Meet Veremin!

Veremin, is a video theremin that allows anyone to make beautiful (:-?) music just by waving their arms! It makes use of TensorFlow.js, PoseNet, as well as the Web MIDI and Web Audio APIs.

Veremin is the brainchild of johncohnvt, from the MIT-IBM Watson AI Lab, who built the first rough prototype. I was then able to whip it into something that really worked!

The application attaches to the video stream from your web camera. PoseNet is used to capture the location of your hands within the video. …

Edited 2019 Mar 11 to include changes introduced in TensorFlow.js 1.0. Additional information about some of these TensorFlow.js 1.0 updates can be found here.

Welcome to part three of a series of posts where I walk you through how TensorFlow.js makes it possible to bring machine learning to the browser. First, there’s an overview of how to bring a pre-trained model into a browser application. Then you’ll find greater detail on how to convert your pre-trained model to a web friendly format. Now in this post, we step through using that web friendly model in a web application.

We continue…

Edited 2019 Mar 11 to include changes introduced in TensorFlow.js 1.0. Additional information about some of these TensorFlow.js 1.0 updates can be found here.

If you’ve been following along, you should already have a high level understanding of how to bring a pre-trained model into a browser application. Now, let’s look at the first steps in this process in greater detail.

Before you can convert a pre-trained model to a web-friendly format and bring it to the browser, you first need a model. A great first model to start learning with is the Image Segmenter from the Model Asset Exchange…

Edited 2019 Mar 11 to include changes introduced in TensorFlow.js 1.0. Additional information about some of these TensorFlow.js 1.0 updates can be found here.

TensorFlow.js brings machine learning and its possibilities to JavaScript. It is an open source library built to create, train, and run machine learning models in the browser (and Node.js).

Training and building complex models can take a considerable amount of resources and time. Some models require massive amounts of data to provide acceptable accuracy. And, if computationally intensive, may require hours or days of training to complete. …

The New Kid on the Block

Previously, when using PixieDust to visualize your data, you first needed to shape the data by manipulating the DataFrame and then launch the visualization. As you explore the data, often you need to remove some data points and visualize only a subset. To accomplish this, you had to go back to your data, reshape it, relaunch PixieDust display with the updated data, and repeat this process until you get the visualization you desire.

With the introduction of Filter in PixieDust, this old workflow is made a little easier! In PixieDust 1.1.7 and later, you can define a filter while exploring…

When we don’t embrace Offline First for web and mobile application development, we let our users down. It’s frustrating when their favorite applications become unusable when disconnected or when operating under terrible network conditions. But much of the pain caused by unreliable internet connectivity can be alleviated with an Offline First approach.

Maintaining network availability and reliability can be difficult. Improving user experience, despite network conditions, doesn’t have to be.

Learning By Example

Developing offline-capable applications can be easier than you think, and our developer advocacy team at IBM has been working on a series of Offline First Shopping List sample applications to…

Previously, you learned how to create your own custom visualization for PixieDust. The visualization code was written and tested directly in notebook cells, which is great for proof-of-concept and rapid iterating. However, it may not be the best vehicle for packaging and distributing your work to a large audience.

Magic is best when it can be experienced by everyone. To get your visualization out there, you will need to take it from notebook cells to a Python module, and then to a software repository for others to download and use.

The props

To make sure your PixieDust magic is more easily consumable…

PixieDust, the open source Python helper library that extends the usability of notebooks, is quite adept at creating magic in notebooks. And with multiple visualizations available, there is magic to be had for just about any situation.

However, for the times when the default visualizations are just not quite what you are looking for, there are options. Sure, you can submit an enhancement request to get new visualizations into PixieDust, but why not get ahead of the game and try to create one yourself!

Most magic tends to be secretive and not readily shared, but PixieDust is open to all…