Taking a snapshot to migrate tasks between a computer and a mobile phone
Synchronizing data between a computer and a mobile phone has generally required connecting the two devices via a USB cable. For simple tasks many people even resort to manually typing text from one device onto another. Apple’s iCloud is designed to take the hassle out of this task by automatically syncing data between your various devices over Wi-Fi, but MIT graduate student Tsung-Hsiang Chang and Google employee Yang Li have developed a system called Deep Shot that makes it possible to transfer simple computing tasks between devices simply by taking a photo of the computer screen with a smartphone’s camera.
Designed to work with Web applications, Deep Shot exploits the fact many Web apps use a standard format called the uniform resource identifier (URI) to describe their current state. URI’s are those long links that contain extra information, such as the start and end points and geographical coordinates in Google Maps, for example.
While these links can be copied and pasted and emailed, Deep Shot simplifies things by sending the URI between two devices over Wi-Fi via software installed on both the phone and all the computers with which the phone will interact.
The camera comes into play when uploading data to the phone by identifying the application open on the screen using existing computer vision algorithms. It is also used to identify the specific computer the camera is trained on – work or home, for example – when downloading data from the phone to a computer. The system will also resize the application window to match the framing of the photo.