• Print
  • Email
    Email
    Close

    Please enter a valid email address

    Please enter a valid email address

    Important legal information about the email you will be sending. By using this service, you agree to input your real email address and only send it to people you know. It is a violation of law in some jurisdictions to falsely identify yourself in an email. All information you provide will be used by Fidelity solely for the purpose of sending the email on your behalf. The subject line of the email you send will be "Fidelity.com: "

    Email
    Close

    Your email has been sent.

  • Share
    Share
    Close

    Mutual Funds and Mutual Fund Investing - Fidelity Investments

    Clicking a link will open a new window.

    • Facebook
    • Twitter
    • LinkedIn
FCAT home
☰
Our Story Our Work Observations Join Us
 
3 MIN EST READ
Emerging Tech Proof of Concept
My Phone is My Interface
By: Christian Laugenberg | May 18, 2020
Coronavirus pathogens can reportedly live on surfaces for up to 72 hours, so minimizing the frequency with which we touch things is paramount. This applies to everyday surfaces such as elevator buttons, door handles and touch screen interfaces. FCAT’s Emerging Technology incubator team developed a Proof of Concept that could help customers and associates control a touch screen, without touching it.

The Challenge

Touch screens are all around us. For example, FCAT uses touch screens at the visitor center to demonstrate our work. Given the new health guidelines around touching surfaces, we needed to find an alternative way for people to interact with the content.

The Value

There are a number of interesting advantages with this technology.

  1. The hardware costs are minimal in that you can operate this setup with a standard computer, display, and mic.
  2. No need to download an app (for this simple use case. More complex use cases with bilateral communication would likely require an app).
  3. No bluetooth wifi or network handshake required

What we Learned

We learned that ultrasound can be used to control a remote screen through a simple web interface. The use of a web application has its limitations. Although two way communication between devices is possible, keeping the devices in sync would require deeper integration if both are dynamically changing using sound as triggers.

Team & Tech

As a potential solution to the problem, the Emerging Tech team explored ultrasonic data transmission. The idea is to use sound signals to connect a smartphone with a touch screen, turning the smartphone into a remote control without the need to download an application or require the remote device to pair with the display.

The user scans a QR-code on the display with a smartphone camera (or QR-code reader) to load a webpage. The site has buttons, which the user can use to interact with the display. When the user interacts with the display by pressing the buttons, high frequency sound signals are played. The display “hears” the sounds and reacts accordingly. This way the only screen you “touch” is your own personal device.

Next Steps

The Emerging Technology team will continue experimenting with ultrasonic data transmission and explore aspects such as interference, multi-user scenarios or the ability to handle more involved and complex interactions with the companion device. The team will also explore various other libraries and offerings to find the best technology to support these experiences.

  • Email
  • Facebook.
  • Twitter.
  • LinkedIn.
  • Print

The Deep Dive

Touch screens are all around us, in the wake of the COVID-19 pandemic people will likely think twice about interacting with a public touchscreen interface. A new solution is needed.

Above is a sample experience where a customer visits one of our visitor centers. The journey suggests that the user approaches the screen and scans a QR-code displayed on the touch screen with a QR-code reader or iPhone camera. The phone opens a website, that shows the buttons, which the user would typically have needed to press on the touch screen. In this setup, however, the user interacts with the touch screen by pressing the buttons on his or her smartphone, which sends high frequency sound signals to the screen in order to change the page.

For this initial POC, the team uses an open source library called ‘quiet-js’ in order to transmit data with sound using web audio. This setup does not require installing any software as the sending and receiving of the ultrasound signals can be handled via web-services.

Using sound signals as a means of connecting a smartphone to a screen is a much less intrusive and complicated process compared to Bluetooth or WiFi . This connection does not require any action by the user such as setting the phone into pairing mode or selecting a WiFi device. All the user has to do is to scan the QR-code.

Please see below a short demo video.

FCAT - My phone is my interface
 
  • Email
  • Facebook.
  • Twitter.
  • LinkedIn.
  • Print
937497.1.0
close

Please enter a valid e-mail address
Please enter a valid e-mail address
Important legal information about the e-mail you will be sending. By using this service, you agree to input your real e-mail address and only send it to people you know. It is a violation of law in some jurisdictions to falsely identify yourself in an e-mail. All information you provide will be used by Fidelity solely for the purpose of sending the e-mail on your behalf.The subject line of the e-mail you send will be "Fidelity.com: "

Your e-mail has been sent.
close

Your e-mail has been sent.
Connect with us.
Follow us.
Read us.

This website is operated by Fidelity Center for Applied Technology (FCAT)SM which is part of Fidelity Labs, LLC (“Fidelity Labs”), a Fidelity Investments company. FCAT experiments with and provides innovative products, services, content and tools, as a service to its affiliates and as a subsidiary of FMR LLC. Based on user reaction and input, FCAT is better able to engage in technology research and planning for the Fidelity family of companies. FCATalyst.com is independent of fidelity.com. Unless otherwise indicated, the information and items published on this web site are provided by FCAT and are not intended to provide tax, legal, insurance or investment advice and should not be construed as an offer to sell, a solicitation of an offer to buy, or a recommendation for any security by any Fidelity entity or any third-party. In circumstances where FCAT is making available either a product or service of an affiliate through this site, the affiliated company will be identified. Third party trademarks appearing herein are the property of their respective owners. All other trademarks are the property of FMR LLC.


245 Summer St, Boston MA

© 2008-2020 FMR LLC All right reserved | FCATalyst.com


Terms of Use | Privacy | Security | DAT Support