P5.js and web-based tests

Besides testing the shellphone idea in Unity, I’ve also continued testing out browser-based alternatives for the design experiments. Today I’ll be going over P5.js, a Processing-inspired library for interactive coding in Javascript. In going through the library functions, I’ll also use the opportunity to try out and learn some of the mobile browser capabilities.

By creating a localhost server on my PC and connecting the mobile to it via WiFi it was easy to create a quick workflow for testing purposes. Every time I save the code, the mobile browser reloads and runs the p5 script automatically.

Next step was to make sure that the main sensors that are available via the core p5 library are working well. Here are a few notes:

  • Acceleration variables responds well;
  • deviceTurned() is somewhat unresponsive;
  • Touch-based interactions seem to work well for touchdown, but not so much for touch-release. This might be an issue for some interactions. Multi-touch works ok, but the tracking of individual touch events is not clearly defined.
  • Rotation controls work well too, with an ok response.
  • Geolocation seems to work well, but needs https protocol, which is a bit more work to setup.
  • The vibration API also works ok.
  • Proximity and light sensors are only available in Firefox, which quite limits their use.

A potential problem with using mobile-browser based tech is that usually the phone sleeps after sometime of inactivity, turning off the screen and sensors input. So, I’ve found a library called NoSleep that blocks that functionality, while requiring an initial gesture to keep the screen awake.

Testing npm as basis for workflow

An important advantage to working with JS is the huge amount of libraries that are available for use. To make it easier to do such integrations, package managers such as npm or browserify are quite helpful. In order to use them, I had to learn how to prepare the p5 sketch to load using that workflow. The code for those tests are on the p5-npm folder on the repo. Basically, this meant using the “instance” mode of p5, so that all variables and functions are contained into a scope that can then be bundled up by browserify.

ToneJS, web audio and latency

A problem that showed up, especially when trying to get input to synchronize with sound, was that Android has very inconsistent latency for different browsers. I tried a few ways to fix latency in Chrome, but none worked. As soon as I tried the mobile version of Firefox, however, the latency was much better. I guess I’ll migrate my development for that browser for now.

On the other hand, after some more tests, it seems that the audio setup works only intermittently on firefox. I might need to add another library / initial step to make sure that the audio context is working as it should.