Q Card for Tom Petty

What can you do with that Thing?

By Conversation, Knowledge, User Interface, Voice Search No Comments

Often, when you have something to do, you start by searching for information about a particular Thing. Sometimes, you know exactly what that Thing is, but often, you find the Thing by using information related to it. 

“Who is Taylor Swift?” → Taylor Swift

“Who directed Avatar”  → “James Cameron”

The “Thing” is what we call a Knowledge Entity and something that you can do with that Thing is what we call a Downstream Action. The bond between that Knowledge Entity and the Downstream Action is what we refer to as Actionable Knowledge.

Actionable Knowledge

How do we do this? Our Knowledge database holds information about all kinds of entities such as movies, TV series, athletes, corporations etc. These Entities have rich semantic structure; we have detailed information about the different attributes of these Entities along with the Actions one can perform on those entities. An Action may be generic (watch a show), but can also be explicitly connected to a mobile app or service (watch the show on Disney+). This knowledge allows the user to follow up on an Entity command with an Action. 

For example, asking a question such as “How tall is Tom Brady?”  allows you to get his height i.e., 6’ 4” or 1.93 metres (based on the Locale of who’s asking) since Knowledge captures these important attributes about Tom Brady. Note that these attributes are different for different types of Entities. That is determined by the Schema of the Entity, which allows validation, normalization and transformation of data.

A command like “Who is Tom Brady?” returns a Q Card with information about Tom Brady, as shown below. As there may be multiple entities referring to “Tom Brady”, a popularity measure is computed so that the correct Tom Brady is returned, based on popularity, context and your current session. Popularity is a special attribute that is computed from multiple attributes of the entity. An Entity Card surfaces the various attributes associated with the attribute, such as when Tom Brady was born, how tall and heavy he is, and what sport he plays. There are also attributes that define potential Actions that can follow, so “go to his Instagram” will instantly take you to Tom Brady’s account in the Instagram app. 

Q Card for Tom Brady

Actions are about getting things done! Here’s another example of being able to instantly go from information to Action using Actionable Knowledge.  Asking “Who is Tom Petty?” followed by a command “listen to him on Spotify” will start playing his music. This is a powerful feature that provides a great user experience and rapid Time to Action® .

Q Card for Tom Petty

The three pillars of the Aiqudo’s Q Actions Platform allow us to implement downstream Actions:

  1. Semantically rich Entities in Actionable Knowledge
  2. AI-based Search
  3. Powerful Action execution engine for mobile apps and cloud services

AI Search

We are not limited by just the name of the entity. Our AI-based search allows you to find entities using various attributes of the entity. For example, you can search for stock information by saying “How is Tesla stock doing today?” or “Show me TSLA stock price”.   Aiqudo understands both the corporation name or the stock ticker when it needs to find information on a company’s stock price.  Some apps like Yahoo Finance can only understand the stock ticker; it may not be built to accept the name of the company as an input. Our platform allows us to fill this gap by decoupling action execution from search intent detection. A middle-tier federation module acts as a bridge between intent extraction and Action execution by extracting the right attributes of the Entity returned by the search to those required by the Action execution engine. In the above example it extracts the stockTicker attribute (TSLA),  from the corporation entity retrieved by the search (Tesla) and feeds it to the Action engine. 

Q Card for Tesla Stock

Voila! Job done!

So, what can you do with that Thing? Well, you can instantly perform a meaningful Action on it using the apps on your mobile phone. In the example above, you can jump to Yahoo News to get the latest finance news about Tesla, or go to the stock quote screen within E*Trade, the app you use and trust, to buy Tesla shares and make some money!

Mobile Accessibility

Accessibility plus utility plus convenience!

By Digital Assistants, User Interface, Voice Search No Comments

It’s great to see various platforms announce specific accessibility features on this Global Accessibility Awareness Day.

A feature that caught our attention today was Google’s Assistant-powered Action Blocks.

It’s a new app that allows users to create simple shortcuts to Actions they commonly perform. They are powered by Google Assistant, but allow for invocation through a tap.

My Actions and Favorites

We built this functionality into Aiqudo’s Q Actions when we launched it in 2017. Our approach is different in several ways:

  • The user does not need to do any work, Q Actions does it automatically for the user
  • Q Actions builds these dynamically – your most recently used Actions, and your favorite ones are automatically tracked for you – you just need to say “show my actions”
  • These handy Action shortcuts are available to you with one swipe to the right in the Q Actions app. One tap to invoke your favorite action. 
  • There’s no new app, just for accessibility – it’s built in to your Assistant interface for convenience – you just need to say “Hello Q, show my Actions”
  • There are hundreds of unique high-utility Actions you can perform that are not available in any other platform, including Google Assistant. Here are a few examples:
    • “whose birthday is it today?” (Facebook)
    • “show my orders” (Amazon, Walmart)
    • “start meditating” (Headspace)
    • “watch the Mandalorian” (Disney+)
    • “watch Fierce Queens” (Quibi)
    • “show my tasks” (Microsoft To Do, Google Tasks)
    • “show my account balances” (Etrade)
    • “join my meeting with my camera off” (Google Hangouts, Zoom)
    • “call Mark” (Whatsapp, Messenger, Teams, Slack,…)
    • “send money to John” (PayPal)
    • . and on and on and on…

It’s just easier, better and more powerful! 

And available to everyone!

Bark to Action

Bark to Action

By Conversation, Startup Culture No Comments

Mike and his Corgi

“It’s a dog’s life!” – Aiqudo work from home #4

Every Friday afternoon, during this lockdown, the Silicon Valley Team gets together for “Social Hour”. We can only assume that the Belfast team is more focused on Irish Whiskey and Guinness on Friday evenings!

Online meetings are not a new thing for Aiqudo. We’ve been doing this twice a week for our entire global team since Belfast came on board in mid-2017!! We’re experts at effective virtual meetings.

And … unlike other startups that just place cute photos of their dogs on Teams’ pages on their websites, our dogs actually participate and provide their opinions during our meetings. Their barks have bite!!

Happy Friday!
#BarkToAction

Monopoly top hat

Aiqudo + Banking = Voice To Transaction

By App Actions, Voice, Voice Search No Comments

We all remember playing playing the game Monopoly as kids right? Well I recently stumbled upon a version of the game that uses voice commands to control the game flow and act as the “bank” – a role most of us avoided so we wouldn’t have to deal with all the annoying transactions such as selling properties and buildings, collecting taxes, exchanging currency, and paying out people as they passed “GO”. Admittedly, it’s a novel use for voice commands in a classic game. But what about using voice to manage apps controlling our real money?

Voice to power more than games

Back in June of last year, we wrote a blog post that talked about the power of voice to perform activities in mobile banking apps. The post specifically referenced Bank of America’s Erica virtual voice assistant as a tool to help users accomplish common, often time-consuming banking activities without the need to memorize complex menus or worse, speak to the dreaded online customer service representative.  The net result of this; a simple, pleasant, user experience that builds brand loyalty and customer retention.  

Enter Aiqudo Voice to Action®

Well, that got me thinking. I’ve been an E*Trade banking customer for years and all this time I’ve never really used voice to make payments, transfer money or check balances. 

Nonetheless, I decided to see if I could recreate and hopefully improve upon my previous experience – this time using our very own Q Actions app.  The following video highlights some of my efforts. 

But can I trust this new way of banking?

Yes. You may have noticed in the video that I am not providing credentials to access my account in E*Trade.  That’s because I’m already authenticated. Previous to shooting the video, I had provided credentials, by way of fingerprint biometric, as part of the very first action execution.  Note that Aiqudo did not manage this process; it was handled completely by the mobile app. And because of this, the data used to hold the credential, lives entirely in the app itself and is neither passed to nor processed by Aiqudo systems at all.  This separation of duties maintains privacy of user data and hence increases trust in using the technology.

A personalized experience

Personalization is a word typically used to describe how an app or other system function adjusts to provide an experience tailored specifically to a user.  It’s often used in conjunction with AI and machine learning systems as the end result of acquiring, processing and suggesting courses of action or data upon which to act.  We enable personalization in the previous actions a couple of ways. If you have, say, more than one voice-enabled banking app on your mobile device (similar to what I have), our system can be configured to remember the user’s preferred app action.  For instance, if I were to say the command “check my balances” Aiqudo suggests actions from both E*Trade AND Wells Fargo. If I choose the E*Trade action, the next time I say the command it will remember E*Trade and perform the action right away – no need to ask again.  Likewise, whenever the action requires the user to provide input such as account number or payee, the system can store these away for subsequent use. These are simple examples but add a nice touch to an already-useful integration. 

What if I don’t bank with E*Trade? What can I do with other apps and is it safe?

Aiqudo maintains similar actions for apps like Venmo and Paypal that allow “send money to <username>” type actions. In each of these cases, Aiqudo defers authentication to the app before completing the transaction and also ensures that the data used by the action in the app, e.g., the payee’s phone number or email address never leaves the device or the app. The following video illustrates this.

With the proper integration of our ActionKit SDK into a banking app such as E*Trade,  the end user reaps the benefits of a trusted, highly- useful voice-powered interface that enables complex and often multi-step operations with ease and reduces Time to Action® for many activities within the app.

Actionable Knowledge

Q Actions 2.4: “Under the Hood” improvements for Productivity and Utility

By App Actions, Artificial Intelligence, Digital Assistants, Voice, Voice Search No Comments

Q Actions 2.4 now available on Google Play

The recent release of Q Actions 2.4 emphasizes Aiqudo’s focus on productivity and utility through voice. As voice assistants are becoming an increasingly ubiquitous part of our daily lives, Aiqudo aims to empower users to get things done. Many of the improvements and enhancements are “under the hood” – we’ve increased personalization and expanded the knowledge that drives our Actions.

Actionable KnowledgeTM

Our content-rich Q Cards leverage Actionable Knowledge to extend functionality into popular 3rd party apps. Start by asking about an artist, music group, sports athlete, or celebrity: “who is Tom Hanks. Aiqudo’s Q Card not only presents information about the actor, but will ask “what next?”. You say “view his Twitter account” or “go to his Instagram”, Actionable Knowledge will drop you exactly where you want to go!

Sample Actionable Knowledge Flow:

  • Ask “who is Taylor Swift?”
  • Select one of the supported Actionable Knowledge apps
    • “listen to her on Spotify”
    • “go to her Facebook profile”
    • “check out her Instagram”

Personalization … with privacy

Q Actions is already personalized, showing you Action choices based on the apps you already trust. We can now leverage personal data as signals to personalize your experience, while still protecting your privacy. It’s another iteration of our continued focus and dedication to increase productivity and augment utility using voice.  For example, if you checked in to your United Airlines flight, and then, the following day, say “show my boarding pass”, the United Airlines action is promoted to the top – exactly what you’d expect the system to do for you.

Our new Personal Data Manager allows secure optimization for specific apps. If you have a Spotify  playlist called “Beach Vibes”, and you say “play Beach Vibes”, we understand what you want and we will promote your personal playlist over a random public channel by that name. Your playlists are not shipped off the device to our servers, but we can still use the relevant information to short-cut your day!  If “Casimo Caputo” is a friend in Facebook Messenger, Messenger will trump WhatsApp for “tell Casimo Caputo let’s meet for lunch”. But “message Mark Smith let’s play Fifa tonight” brings up WhatsApp since Mark Smith is your WhatsApp buddy.

Simply do more with voice! Q Actions is now available on Google Play.

Aiqudo Auto Mode

Aiqudo and Sensory Collaborate on Holistic Voice Solutions

By App Actions, Auto Mode, News, User Interface, Voice No Comments

Will develop solutions that simplify the process of integrating voice assistants into a variety of devices  

BUSINESSWIRE, LAS VEGAS, January 6, 2020

Aiqudo, a leading voice technology pioneer, today announced that it is collaborating with embedded voice and vision AI leader Sensory to bring to market comprehensive voice solutions that serve as white-label alternatives for voice services and assistants. The two companies are working on solutions targeting automotive, mobile app, smart home and wearable device applications.

Currently, companies and brands must piece together different technologies to fully implement a voice solution. With their technologies combined, Aiqudo and Sensory will deliver a fully integrated end-to-end solution that combines Sensory’s wake word, voice biometrics and natural language recognition technologies with Aiqudo’s multilingual intent understanding and action execution (Q Actions®) to provide the complete Voice to Action® experience consumers expect.

“Voice adoption continues to grow rapidly, and brands are always exploring ways to streamline the process of integrating a convenient voice UX into their products,” said Todd Mozer, Sensory’s CEO. “Working with Aiqudo allows our two companies to provide the industry a turn-key solution for integrating powerful voice assistants into their products that feature brand-specific wake words and are capable of recognizing who is speaking.”

“Users just need to enter the cabin with their smartphones. There’s no registration required, and the personalized wake word and voice biometrics allow users to instantly access their personal apps seamlessly and securely”.

With both Aiqudo and Sensory positioned as leaders in their respective fields, this collaboration is a natural fit as their technologies are highly complementary.  The initial integration is focused on the automotive vertical that will be showcased at 2020 Consumer Electronics Show.

Aiqudo’s Auto Mode highlights a highly personalized user experience in the car using the Q Actions® platform. Enhanced with Sensory’s wake word (TrulyHandsfree™) and voice ID (TrulySecure™) functionality,  multiple users seamlessly access their personal mobile devices just by using their voice to execute personal actions. “Users just need to enter the cabin with their smartphones,” said Rajat Mukherjee, Aiqudo CTO. “There’s no registration required, and the personalized wake word and voice biometrics allow users to instantly access their personal apps seamlessly and securely”.  

“Brands increasingly want to create their own branded voice experiences for their customers,” said John Foster, CEO of Aiqudo. “Working with Sensory, we have created the easiest and fastest way for brands to bring the power and convenience of voice to their customers. We are excited to integrate our areas of practice and expertise to deliver a comprehensive solution.”  

To view the demo onsite while at CES, please email Aiqudo at CES@aiqudo.com. 

BUSINESSWIRE: Aiqudo and Sensory Collaborate on Holistic Voice Solutions

About Sensory

For over 25 years, Sensory has pioneered and developed groundbreaking applications for machine learning and embedded AI – turning those applications into household technologies. Pioneering the concept of always-listening speech recognition more than a decade ago, Sensory’s flexible wake word, small to large vocabulary speech recognition, and natural language understanding technologies are fueling today’s voice revolution. Additionally, its biometric recognition technologies are making everything from unlocking a device to authenticating users for digital transactions faster, safer and more convenient. Sensory’s technologies are widely deployed in numerous markets, including automotive, home appliances, home entertainment, IoT, mobile phones, wearables and more, and have shipped in over two billion units of leading consumer products and software applications.

For more information about this announcement, Sensory or its technologies, visit https://www.sensory.com/, contact sales@sensory.com or for press inquiries contact press@sensory.com.

About Aiqudo

Aiqudo (pronounced: “eye-cue-doe”) is a Voice AI pioneer that connects the nascent world of voice interfaces to the useful, mature world of mobile apps and cloud services through its Voice to Action® platform. It lets people use natural voice commands to execute actions in mobile apps and cloud services across devices. Aiqudo’s SaaS platform uses machine learning (AI) to understand natural-language voice commands and then triggers instant actions via mobile apps, cloud services, or device actions, enabling consumers to get things done quickly and easily.

Aiqudo’s proprietary technology is covered by more than 30 granted patents and patent applications. Aiqudo’s technology is delivered in a scalable approach to creating voice-enabled actions without mandating APIs or developer dependencies.

To see Aiqudo in action, visit Aiqudo’s YouTube channel (youtube.com/aiqudo)

Aiqudo in the Byton Cabin

The BYTON next generation infotainment experience

By App Actions, Auto Mode, Digital Assistants, User Interface, Voice No Comments

Byton Multiple Displays

Peter Mortensen, Solution Architect, BytonPeter Mortensen, Solution Architect, BYTON, Santa Clara, California

BYTON is the automotive EV brand taking the lead in next generation infotainment experiences with a vision of giving vehicle occupants the best combination of safety and entertainment while on the road. The first BYTON is the M-BYTE SUV incorporating a groundbreaking infotainment system with an advanced Android Automotive app platform combined with BYTON’s unique user interface including the industry’s largest display and user control via touch, gesture and voice.

One of the key challenges when expanding the infotainment capabilities in a vehicle is to avoid unsafe distractions for the driver. By extensive support of voice control, the driver in a BYTON has safer operation of the vehicle allowing more visual focus on the driving itself. Another challenge is how well a vehicle’s infotainment system provides the frequently used features of popular apps and online services common on the occupants’ smartphones.

Aiqudo’s Voice to Action® platform enables occupants of a BYTON vehicle a simple and seamless link between natural language voice interaction and their favorite personal apps. With this solution, the occupants of a BYTON can speak naturally to their favorite apps using the vehicle’s powerful microphone array system from any seat in the cabin. The apps can either be residing in the vehicle’s own infotainment system or on the Bluetooth-connected Android or iOS smartphones of the occupants. The Aiqudo solution interprets the spoken commands and automatically identifies which specific app to control. For example, this means the driver can easily send a message using voice through his favorite social networking app installed on his smartphone, hands-free, without picking up the device, thus staying focused on safely driving the vehicle. 

BYTON is committed to intelligently combine the best of voice control with the safe use of touch control, gesture control and display feedback. The Voice to Action® platform expands beyond voice interaction by allowing BYTON’s engineering to touch-enable relevant app controls and provide display feedback when relevant. Furthermore, BYTON’s voice interaction concept aims high by creating a truly intuitive voice interaction experience through integration of Aiqudo’s app control with vehicle control and online digital voice assistant services using Amazon’s Alexa and other digital voice assistant services, depending on country.

More information on our Aiqudo partnership is available at BYTON’s new developer siteAIQUDO and BYTON Partner to enable actions for your phone and car

This post is authored by Peter Mortensen, BYTON Solution Architect. Aiqudo and BYTON announced a strategic partnership at CES 2020 that features voice control of apps in BYTON automobiles using Aiqudo technology.

Aiqudo Voice Platform To Power Digital Experience in BYTON’S Electric Vehicles

By Auto Mode, Digital Assistants, News, Voice No Comments

BUSINESSWIRE, LAS VEGAS, JANUARY 5, 2020

Aiqudo in the Byton Cabin

Aiqudo, a voice technology pioneer, announced ahead of CES 2020 a partnership with premium electric vehicle manufacturer BYTON, bringing the power of Aiqudo’s Voice AI platform to BYTON cars. Aiqudo’s Voice to Action®  platform will enable interacting with your favorite apps on your mobile phone hands-free while driving, seamlessly integrating with BYTON’s unique Digital Experience.

Aiqudo’s industry-leading Voice AI platform will voice-enable actions in native apps within the BYTON ecosystem as well as intelligently launch app actions on personal mobile devices in the vehicle.  BYTON drivers and passengers will be able to navigate, make calls, send messages, listen to music, shop, join meetings, make payments and more using simple voice commands with apps they use and love. In its integration with BYTON, Aiqudo incorporates the personalization and individual choice reflected by consumers’ favorite apps, as well as personal elements within apps such as preferred playlists, contacts, or favorites, all without user registration or setup. The BYTON experience powered by Aiqudo delivers the safest, easiest and most useful way to use a mobile device while in the car.

“A seamless voice experience is integral to BYTON’s groundbreaking user experience and Aiqudo Voice will make accessing your favorite apps convenient and safe,” said Jeff Chung, BYTON Vice President of Digital Engineering. “Aiqudo’s white label solution allows us to explore new possibilities with our expanding partnerships in the BYTON digital ecosystem.”

“BYTON’s vision of the car of the future, equipped for autonomous driving, will accelerate the need that users have to access their personal digital lives everywhere. Aiqudo makes this easy!” 

Aiqudo’s voice platform comprises a semiotics-based intent engine that understands natural language commands in 7 languages currently, plus an action execution capability across thousands of applications that consumers rely on daily. The company’s white-label voice platform allows car manufacturers, phone and smart device OEMs and mobile app developers to define unique voice experiences for their customers.  

“Byton has reimagined the relationship between cars and the people who drive or ride in them, placing voice-based interactions at the center of the in-car experience. We believe that voice will soon be the primary way people interact with their digital world. We’re partnering with BYTON to bring a high utility, personalized voice experience to their automobiles,” said John Foster, CEO of Aiqudo. “The in-car experience is a prime use case demonstrating the power of voice. Customers can now drive safely, undistracted and hands-free, and still use their favorite apps just by using their voice.”

Aiqudo’s Action Kit functionality will be offered to app developers through the BYTON developer portal. 

“Action Kit enables BYTON app developers to easily and effortlessly enable voice within their applications for the company’s range of cars and expansive infotainment systems,” said Dr. Rajat Mukherjee, Aiqudo CTO. “BYTON’s vision of the car of the future, equipped for autonomous driving, will accelerate the need that users have to access their personal digital lives everywhere. Aiqudo makes this easy!” 

BUSINESSWIREAiqudo Voice Platform To Power Digital Experience in BYTON’S Electric Vehicles

About Aiqudo

Aiqudo (pronounced: “eye-cue-doe”) is a Voice AI pioneer that connects the nascent world of voice interfaces to the useful, mature world of mobile apps and cloud services through its Voice to Action® platform. It lets people use natural voice commands to execute actions in mobile apps across devices. Aiqudo’s SaaS platform uses machine learning (AI) to understand natural-language voice commands and then triggers instant actions via mobile apps, cloud services, or device actions, enabling consumers to get things done quickly and easily.

Aiqudo’s proprietary technology is covered by more than 30 granted patents and patent applications. Aiqudo’s technology is delivered in a scalable approach to creating voice-enabled actions without mandating APIs or developer dependencies.

To see Aiqudo in action, visit Aiqudo’s YouTube channel (youtube.com/aiqudo)

About BYTON 

BYTON is a global premium electric vehicle manufacturer that is creating the world’s first smart device on wheels. By integrating advanced digital technologies to offer a smart, connected, and comfortable mobility experience, the company is designing an EV that will meet the demands of an increasingly digital lifestyle now and into the future. 

The company’s global headquarters and state-of-the-art manufacturing center are located in Nanjing, China. Its global R&D hub is located in the heart of Silicon Valley and devoted to the development of BYTON’s groundbreaking intelligent car experience, digital ecosystem, advanced connectivity, as well as other cutting-edge technologies. BYTON’s design and concept vehicle center is located in Munich, Germany. 

BYTON’s core management team is made up of top innovators from leading-edge companies such as BMW, Tesla, Google, and Apple. This diverse group of leaders from China, Europe, and the US share the singular vision of creating an unprecedented automotive experience.

Official website: www.byton.com  Further Information: BYTON Newsroom 

Internships: The New Normal

By Natural Language, Startup Culture No Comments

Kenny Kang

Intern Voice: Kenny Kang

Working at a startup can be described as, interesting, but in the best way possible. As a comparison, my summer roommate interned for a larger corporate company and we developed two completely different ideas of a what a ‘normal’ working environment is. Apparently, it isn’t ‘normal’ for an internship project to become a feature in the company’s main product. It also isn’t ‘normal’ to have the opportunity of presenting directly to C suite executives. And it definitely isn’t ‘normal’ to be talking to your CEO about his wild college days during company outings. I could write entire essays about all the reasons I loved working at Aiqudo, but there was one that always made my friends question my sanity: I kept describing the work itself as ‘fun!’. Even for a startup, I’m not sure how normal that is.

During my internship, I created a question answering service which dealt with knowledge-based queries. For example, questions like “How old is Tom Brady?” or “Which movies were Lawrence Fishburne and Keanu Reeves in together?”. While the problem itself was interesting, it was the freedom I had that made it really engaging. Since there isn’t always a straightforward solution when dealing with Natural Language Processing (NLP) problems, I needed to constantly approach the next obstacle in new ways, such as using certain tools in unconventional ways or reading up on the latest research. Each new day felt like solving a new puzzle, and that’s what made it consistently so enjoyable! Of course, I ran into plenty of issues that seemed impossible to get around, but luckily, I had an amazing mentor, Sunil, who was always there to point me in the right direction.

This past summer has been an incredible experience. I came in thinking I would leave with a few new skills. Not only have I learned several valuable skills, I’ve also developed a newfound confidence in my ability to think through complex problems, and set a higher bar for any company I’d want to work for in the future.