Creating 3D models with your SmartPhone

Creating a 3D digital representation of a real life physical object seemed like an impossible task for an amateur. But now this has become quite simple for anyone with a smartphone.

I tried my hand at this using 123D Catch (by Autodesk)  to explore the ease of use, quality of output and ability to share and export.  

Here is the link

To view the 3D rendering

1. Click on the image or link above
2. On the page that loads, click on 3D view (at the bottom of the image)
3. You can use your mouse for navigation control (click on "?" for instructions or see below)

What did I learn from this exercise?

1. The process was quite simple - all you need is the app and a smartphone
2. Make sure you take shots from about the same distance, with good lighting and follow the grid on the app.  It is a lot like photosphere or other similar apps
3. The upload and processing takes significant amount of time.  About 1-2 hours.
4. Once done, you can download the files for the model (*stl) 
5. You can make your model public and share with others.  You can tweet, Pin, etc.  You also get an embed code which for some reason did not work for this blog post.

Next steps:

Explore use of this tool for anatomy education!

Tips for navigation

1. Zoom in to the drummer - on touch pad you may have to use 2 finger slide, on mouse use scroll wheel
2. Pan to center the drummer - Shift + left click + Drag (slide on touch pad)
3. Rotate - Left Click + Drag (slide)

Google Photos - Machine Learning that is Insanely Good!

Google just released the new version of Google Photos for desktop, Android and iOS and after using it for just a few minutes I am completely blown away.

While it is many great features, the "search for photographs" is what took my breath away.  I started off searching for "Wedding", "Beach" etc and then moved on to specific landmarks like "Taj Mahal" and "Chichentiza" and it performed remarkably well.

Then I tried "cricket" and the results were stunning.
Here is a sample of what it found in my photo library:

1.  This was not a big deal - it had tons of clues that it was a cricket match

This was from a famous test match in England

2.  The next one was a screen capture from a smartphone of a scorecard from a cricket match

This had the word "Cricket" in the photo

 3.  This was when I started to really be amazed.  This was a screen capture I took of 3 women from Afghanistan celebrating a famous victory.  The only hint here was the score at the bottom.

Afghan women celebrating a famous victory

 4.  This one had few background clues to this being a cricket match.  It is possible since I had grabbed this from the web that Google had indexed it as being a game of cricket.

Cricket match in ice and snow.

5.  This one was totally amazing.  The FIFA trophy would make you think that Google would index this under soccer.  But the 2 people flanking the trophy are 2 famous cricket players.  Google recognized them and brought this up in the search for cricket.  

Tendulkar and Ganguly 

 6.  And lastly - this grainy shot of my TV showing a cricket match - all you can see is the helmet.

Cricket match on TV

Between the amazing experience with Skype translator and this one with Google Photos it seems that machine learning has reached beyond our wildest expectations.

Human Learning Needs in the Age of Machine Learning

Yesterday I experienced quite a coincidence that helped me crystallize my thoughts regarding machine learning; as I was scanning my Twitter and Google+ timelines and my Inoreader feeds each site had a post about machine learning. This post is about 2 common questions I get asked and how these posts helped me reflect on my responses to these questions.

Due to my work with the IBM Watson research team, I often get asked these 2 questions by other physicians:

•  The first question is if/when computers are going to make physicians redundant.  My response is that at present the vision is to create Artificial Intelligence systems that would help healthcare providers provide more efficient or better quality care and not to replace them.  Clearly, no one has a crystal ball and the expert opinions run the gamut from fear of AI to tremendous optimism.

From Newsweek

Peter Diamandis and Steven Kotler

• The second question is how to train our current generation of learners to prepare them for the future where they will work closely with artificial intelligence systems.  My response here is that students need to be part of building these systems in their disciplines.  Just like in high schools students take immersive experiences to learn foreign languages, our learners need to start doing electives in computer sciences and  data analytics departments or companies to leverage these technologies to solve problems in their disciplines.  Thus medical students should work with computer science students to use big data from wearable devices to improve health of a population for example.  This will not only help solve problems but they will learn first hand the limitations of these tools and recognize these in the future rather than just blindly relying on their recommendations.  

So what were the three posts?

1.  The Machine Vision Algorithm Beating Art Historians at Their Own Game

This is a post about a paper by researchers at Rutgers University who developed machine learning algorithms to help recognize styles and artistss of fine art paintings with a great deal of accuracy.  But the most important lesson was that when the algorithms failed to identify an artist correctly, there was a lesson to be learned.  There was a similarity in the paintings that was evidence of how the two artists (the correct one and the one wrongly identified by the algorithm) were similarly influenced.  Something that an art historian may not even have been aware of.

From: MIT Technology Review (Click image for web page)

2. Image Scaling Using Deep Convolutional Neural Networks

This amazing post describes in an (relatively speaking) easy to understand manner how a neural network was designed to process low resolution images to "fill in the missing pixels" to produce high-res images for Flipboard posts.  The post is a great one for our high school and college math students to see how concepts that they are learning have tremendous practical implications.

Courtesy Normal Tasfi via Flipboard

3.  a16z Podcast: Making Sense of Big Data, Machine Learning, and Deep Learning

This is a terrific 27 minute podcast the quotable quote being,

"Machine learning is to big data as human learning is to life experiences" 

It has a great story of how Larry Page talking with Google employees exhorts them to shorten the latency between entering a search term and getting the results.  When asked if what the final goal should be, whether it should be zero, he responds, "Why should we stop at zero?".  The goal should be that machines should be able to anticipate our needs not just respond to our requests.  Nguyen goes on to discuss why machine learning needs to be part of every app.

Sonal interviewing Christopher Nguyen CEO of Adatao

Given enough data, machine learning can identify patterns that humans cannot and will be able to predict problems before then happen.  Thus humans know that driving a car with poor brakes and wipers, very fast in rain on a curvy road often leads to disasters. 

We have an increasing torrent of data from medical literature, genomic analysis, electronic health records and wearable devices. When we start making this data available to appropriately programmed machines, patterns will emerge that may help predict or prevent heart attacks, strokes and cancers.  

This is what our learners need to get comfortable with so they know when to rely on these predictions and when to spot errors and learn from them.

Microsoft Strings Together an Amazing List of Innovations!

This has been an amazing year for Microsoft.  The list of hardware and software innovations that they have come out with is mind boggling.

1. Surface Pro 3 - probably the best single device I have owned - replaced my laptop and iPad and Android tablet in one fell swoop and added the functionality of the superb Stylus.
2. Office Mix - the easiest way to create content for flipped classrooms
3. HoloLens - the mixed reality headset that has huge potential in education
4. Surface Hub - Large "Smartboard" with video conferencing, OneNote and motion sensing built in.
5. Microsoft Band - for $200 a fitness tracker with ability to read notifications, tweets, text messages, email, calendar alerts, heart rate monitor, GPS etc.
6. Windows 10 with Microsoft Edge - a browser that lets you annotate the web
7. Skype Translator (in Preview) - instant translation between English, Spanish, Mandarin, Italian and other languages coming soon.

On top of this making all of Office apps available on both iOS and Android and also allowing developers to quickly convert their iOS and Android apps for Windows is a big shift in MS philosophy.

Competition is great for consumers and I am glad MS is back in the fray in a big way!

HoloLens for Anatomy Education

Office Mix Video

Surface Hub

Web Note feature of Microsoft Edge Browser