JavaScript for… deep learning?

by | Sep 13, 2018 | TensorFlow.js

Would you have a moment to talk about JavaScript?

If you’re anything like me, you never really considered learning JavaScript for deep learning.

Why” — I’d say — “would I learn JavaScript, the language of the web, to do something” — I’d continue — “I can do perfectly fine with Python?”.

I’ve understood and accepted the truth since, and my life changed for the better. I have seen the light.


Web-apps are awesome

Over the past 20 years, web browsers have evolved from clunky applications displaying articles with hypertext and images, into our default environments for work and play. Like it or not, huge chunks of our daily activities happen inside web browsers — whether it’s communication, education, entertainment, or work. Integration of web technology into our daily workflow is so deep it’s sometimes hard to notice it (let alone not take it for granted!).

Let’s take a step back here.

The words you’re reading were written in Google Docs — an in-browser app. Unless you’ve been faxed this article by an uncle — I’m willing to bet you’re reading these words using a web browser. If you’re a data scientist, most of your work is probably done in Python, using various additional tools to make your life easier. You’re using interactive browser apps like Jupyter Notebooks to write, document, and present your work.

Jupyter notebook (a miracle of technology)

If you want to create interactive plots within Python, Bokeh is a go-to option. It allows enriching charts in ways impossible for any static visualization tools — mouseover labels, animations, and so on.

See the interactive Bokeh reproduction of the visualization used in the iconic TED talk by Hans Rosling.

If you use TensorFlow as your deep learning library, chances are you are familiar with using TensorBoard for tracking the training process, visualizing embeddings or computational graphs.

TensorBoard’s Graph Visualization [Source:]

Mentioning these three tools doesn’t even scratch the surface of powerful browser-based apps data scientists benefit from.
All of these tools are charmingly sleek; their output engaging and interactive.
They are all a joy to use, whether you are developing with them, or interacting with the output.
They are all — ahem — they are all written in JavaScript!

So what” — I’d say — “I don’t need to know JavaScript to be able to use these tools with Python! Please stop knocking on my door, trying to talk to me and my family about JavaScript.”

I was missing the whole point. Here’s why you need to ask JavaScript into your data science heart.

Neural networks are hard (to share with people)

In the past 3 years, deep learning frameworks like TensorFlow, PyTorch, and Keras, have sprung like delicious mushrooms after the rain. They have simplified the process of going from an idea to creating a robust statistical model capable of solving a problem in your field. Whether you use deep learning in AI research or art, medicine or astrophysics, entertainment or education: sharing and communicating your models with others is invaluable.

As my pastor says:

“A picture is worth a thousand words; AI-powered web app is worth a thousand and twenty-eight pictures.”

Historically, the process of wrapping your AI model in an in-browser service or experience has been complex and frustrating. This made disseminating the result of deep learning work in an accessible, visually pleasing form an additional, discouraging roadblock. Think of all the good that could be done with more effective means of scientific communication (think: more distill.pubs around). It’s hard to overstate how much more engaging the experience of absorbing a deep learning paper is, when you are able to interact with the very model it’s describing. Think of all the useful (educational or otherwise) applications of deep learning that exist merely as code on GitHub and never got an interactive frontend to attract a wider audience.


Interactive means…yummy

“The interactivity…” — the pastor would often add — “ is the missing cherry on top of a righteous linear algebra cake”.

The Neural Network Playground by Google is a well-known example of an interactive deep learning demo. I remember the joy of playing with these interactive examples while getting a grasp of the basic concepts in machine learning. I know there are many other data scientists, whose interest in deep learning got ignited and propelled by this, and similar demos.

TensorBoard’s Graph Visualization [Source:]

When it comes to AI, whether it’s an educational demo, a little scavenger hunt game, a part of a research project on t-SNE, or a new musical instrument, interactivity allows the user to be more engaged with the experience. Interactivity can turn a passive viewing experience into a playful, memorable use of user’s attention. What’s cool about in-browser AI is that “interactivity” here is not limited to pointing and clicking at things: users can feed locally run models with various sources of input like their camera or microphone.

Abhishek Singh interacting with his app, which allows talking to Amazon’s Alexa using simple sign language commands.

The possibilities for innovative applications of in-browser AI seem endless. See our growing repository of interactive applications of machine learning for more inspiring examples.


In-browser AI has never been simpler (or safer)

Interest in using JavaScript for deep learning has been gaining momentum recently, but it’s certainly not a new idea. In fact, it may be older than many of us would guess. Some internet archeologists date first pioneering attempts by Andrej Karpathy as far back as June 2014. That’s a very long time ago indeed, in the fast-moving world of deep learning. What has changed since?

There were multiple early attempts at making a JavaScript framework for deep learning, most notably Keras.js and TensorFire. More recently, our community has been blessed with TensorFlow.js (TF.js) and Ml5.js (the latter being a high-level wrapper for TF.js). These frameworks can be used to define, train, and run deep learning models directly in the browser. Developing models in TF.js can be done at various levels of abstraction, from the raw TensorFlow.js API, to the more forgiving ‘layers API’. The latter is modeled after Keras (for Python), with its Lego-like blocks for deep learning. Ml5.js is even more accessible — it was made with artists and creative coders in mind. The best thing is — In-browser AI apps made with TensorFlow.js and Ml5.js don’t require users to install anything. Everything is ready to go, right there, on a silver platter of a browser tab. Models can be run and fine-tuned on client’s end — this means users don’t need to trust their data with anyone. As for the (optional and seamless) GPU support — it’s accommodated by WebGL, and not limited to Nvidia cards. Whoosh!


Blessed by JavaScript

As you can see, everyone can benefit from the ability to make interactive in-browser AI apps. It’s never been easier. All it takes is to peak out of your comfort zone of Python and embrace JavaScript.

Once you do, make sure to spread the word and share your deep learning models — with millions of people on the internet — or only a few that matter to you.

Author’s note: A sincere thank you to Piotr Migdał for his invaluable guidance and being generous with his time, which made this article possible.

Rafał Jakubanis

Rafał Jakubanis

Data scientist at Interested in art, artificial neural networks, and peanut butter.