Frankenstein and the future networked car

So what do Frankenstein and the future networked car have in common, you ask? Simple: both are compelling stories brought to life in Geneva, Switzerland.

In Mary Shelley’s Frankenstein the creature is seen climbing Mont-Salève after having fled Geneva during a lightning storm:

“I thought of pursuing the devil; but it would have been in vain, for another flash discovered him to me hanging among the rocks of the nearly perpendicular ascent of Mont-Salève.”

Mont-Salève, overlooking Geneva
Photo: Benoit Kornmann

Of course, the future networked car is a very different type of story, but compelling nonetheless. The laboratory in this story is the ITU Symposium on The Future Networked Car being held within the Geneva Auto Show on March 5 to 6, where many new ideas will be brought to life by convening leaders and technical experts from the automotive and ICT communities.

The event, organized by the International Telecommunications Union (ITU), will consist of high-level dialogues and several technical sessions; these include a session on integrating nomadic devices in cars, where I will discuss how technology standards can help minimize driver distraction. The dialogues will cover road safety and innovation for the future car, and will feature key leaders such as the presidents of Fédération Internationale de l’Automobile (Jean Todt) and Infiniti (Johan de Nysschen). The technical sessions will explore automated driving, connected car use cases, emergency services, and, of course, nomadic device integration. Speakers for these sessions come from a mix of automakers, tier one suppliers, ICT companies, standards development organizations (SDOs), industry groups, and government agencies.

The symposium also includes a session jointly organized by the ITU and UNECE Inland Transport Committee that deals with the human factors and regulatory issues introduced by automated driving. This session is an encouraging sign that the ITU and UNECE will continue the collaboration they started last June (see my previous post, “UN agencies take major step towards international standards for driver distraction”).

Hope to see you in Geneva!

QNX acoustics technology shortlisted for 2014 embedded AWARD

Okay, first things first. I didn't get the capitalization wrong. The name of the award really is spelled that way. I thought it odd at first, but I'm getting used to it. And besides, who am I to complain? After all, I spend a good part of my life promoting a product whose name is spelled all uppercase, and... where was I? Oh yes, the award!

Every year, the folks who organize the embedded world Exhibition&Conference hold the embedded AWARDs, which honor the most innovative software, hardware, and tools for embedded developers. And this year, the competition judges selected QNX Acoustics for Active Noise Control as a finalist in the software category.

If you aren’t familiar with our ANC solution, allow me to provide an overview — which will also help explain why the embedded AWARD judges are so impressed.

Automakers need to reduce fuel consumption. And to do that, they employ techniques such as variable engine displacement and operating the engine at lower RPM. These techniques may save gas, but they also result in "boom" noise that permeates the car's interior and can lead to driver distraction. And who needs more distraction?

QNX Acoustics for Active Noise Control can integrate 

seamlessly into a vehicle's infotainment system.

To reduce this noise, automakers use ANC, which plays “anti-noise” (sound proportional but inverted to the offending engine tones) over the car's speakers. The problem is, existing ANC systems require dedicated hardware, which adds design complexity, not to mention significant Bill of Materials costs. And who needs more costs?

Enter QNX Acoustics for ANC. Rather than use dedicated hardware, QNX ANC provides a software library that can run on the existing DSP or CPU of the car's head unit or audio system. This approach not only reduces hardware costs, also enables better performance, faster development, and more design flexibility. I could go on, but I will let my colleague Tina Jeffrey provide the full skinny.

Did I mention? This wouldn’t be the first time QNX Software Systems is tapped for an embedded AWARD. It has won two so far, in 2006 and 2004, for innovations in multi-core and power-management technology. It was also a finalist in 2010, for its persistent publish/subscribe messaging. Here's to making it a hat trick.

The QNX sound machine at CES

If you’ve ever had the pleasure of attending the Consumer Electronics Show, you’ll know that it’s a crowded place full of lights and noise. In the automotive North Hall, much of the cacophony comes from the legions of car customizers blasting bass from sedan-sized speakers. This year, QNX has brought a new kind of technology concept car to CES, based on a Kia Soul, that offers some subtler forms of sound artistry. (Sorry, hamster fans—I don’t think we’ll have your favorite mascot in the QNX booth.)


A sound ride: the new QNX technology concept car for acoustics

Let’s start with noise. Everyone likes a booming radio, sometimes. But if that’s the only tool you have to drown out engine noise you’ll go deaf. That’s where Active Noise Control (ANC) comes in. Think of ANC as a more sophisticated version of noise cancelling headphones that you don’t need to wear. Not only does ANC help keep the car’s cabin quiet, but the QNX solution is software based and doesn’t require a dedicated hardware module, saving the OEM and the consumer money.

The best part about ANC is that it helps cars become more fuel efficient. Huh? To keep car interiors quiet, automakers add baffling in the doors and under the floor to help mute engine noise. Dragging around that extra weight costs fuel. So removing the ballast (I mean baffles) lets the automakers make more fuel-efficient cars. And with ANC, which helps eliminate the extra noise caused by this approach, everyone wins.

Beyond wideband
Next up: a new level of call quality. If you’ve had the pleasure of conversing between two newer smartphones (BlackBerry Z10 or Z30, iPhone 5, Nokia Lumina 520, Samsung Galaxy S4, ...) you may have noticed that the call sounded better than what you’re used to. That’s because many newer phones support something called wideband audio (or HD Voice), which transmits more audible frequencies to make the call sound clearer. That’s good, but QNX wants to show what’s possible beyond wideband. So in the QNX technology concept car for acoustics, we’re demoing a new audio feature called full-band stereo calling, which is like having phone calls with CD quality audio. A full-band call has over six times the transmitted frequency range of a standard call, and more than double that of wideband. And as the name suggests, full-band stereo provides two independent channels, adding to the depth and sense of presence, making the call quality something that just has to be experienced.

Sound like a V8, sip like a Volt
Lastly — we get to pump up the volume! The technology concept car for acoustics also sports engine sound enhancement (ESE), which plays synthesized engine sounds over speakers inside the car. With ESE, your engine appears to sound a little more throaty. It may not be obvious, but this is also a fuel saving technology! As carmakers look for creative ways to turn gasoline slurpers into sippers, they’re implementing technologies that dynamically modify engine cylinder firing. Those changes can sometimes make a perfectly powerful engine sound anemic, which negatively impacts customer first impressions. Unfortunately, most people want a car that sounds and performs like it has a huge V8 even if they expect it to sip gas like a Chevy Volt. Both ANC and ESE can help the customer get over their performance anxiety. ESE also lets drivers get in tune with their engine, making it easier to shift by ear.

If you’re up for a little fun, you can also use ESE to make your car sound like something completely different. We’re playing the ESE audio outside the car as well as inside it. The Kia is using QNX ESE audio to masquerade as another car. Tweet us at @QNX_Auto if you can guess what it is!

More QNX-powered cars and infotainment systems from 2011 CES

The second installment in our CES Cars of Fame series. Today, we look at several systems from the 2011 CES event, starting with this week's inductee, a BMW Z4.

Paul Leroux

I've led you astray — sort of. Last week I stated that the LTE Connected Car, the first QNX-powered technology concept car, appeared at 2011 CES. But I didn't mention that QNX technology was at the core of several other innovative vehicles and infotainment systems at CES that year.

So let me set the record straight. And the best place to start is the QNX booth at 2011 CES, where a BMW Z4 roadster was the front-and-center attraction.

BMW Z4 Roadster with ConnectedDrive
The Z4 wasn't a technology concept car, but a true production car straight off the dealer lot. It was equipped with the QNX-based BMW ConnectedDrive system, which offers real-time traffic information, automatic emergency calling, and a text-to-speech feature that can read aloud emails, appointments, text messages, and other information from Bluetooth smartphones. It's a cool system right at home in this equally cool cockpit:



Heck, the whole car was cool, from the wheels up:



Audi A8 with Google Earth
Mind you, the coolness didn't stop at the QNX booth. Just down the hall, Audi showcased an A8 sedan equipped with the QNX-based 3G MMI infotainment system, featuring Google Earth. This same model drove home with the 2011 Edmunds Breakthrough Technology award a short while later.

I don't have any photos of the Audi from the CES show floor, but if you head over to the On Q blog, you can see some snaps from an automotive event that QNX hosted in Stuttgart two months earlier. The photos highlight the A8's innovative touchpad, which lets you input destination names by tracing them with your finger.

Toyota Entune infotainment system
And now to another award-winning QNX-based system. Toyota Entune embraces a simple, yet hard-to-achieve concept: help drivers interact with mobile content and applications in a non-distracting, handsfree fashion. For instance, if you are searching for a nearby restaurant, Entune lets you ask for it in a conversational fashion; no need for specific voice commands.

You could tell the judges for the CNET Best of CES awards were impressed, because they awarded Entune first prize, in the Car Tech category — the first of three QNX-powered systems to do. QNX Software Systems went on to win in 2011 for its QNX CAR Platform and then Chevy won in 2012 for its MyLink system. Not too shabby.

A cluster of clusters
We've looked at just three of the many QNX-based automotive systems showcased at 2011 CES. For instance, QNX also demonstrated digital instrument clusters built by Visteon for the Land Rover Range Rover and for the Jaguar XJ sedan, below:



Freescale, NVIDIA, TeleNav, and Texas Instruments also got into the act, demonstrating QNX systems in their booths and meeting areas.

Do you have any memories of 2011 CES? I'd love to hear them.

Distracted driving — the stats are alarming

I was driving to work the other day when I heard something on the radio that almost made me drop my smartphone. The Ontario Provincial Police (OPP) announced that, for the first time, deaths attributable to driver distraction outnumber those caused by impaired driving. So far this year, on roads patrolled by the OPP, distraction has led to 47 deaths, while impaired driving has led to 32.

This stat drives home the need for dramatically better head-unit integration of services that drivers would otherwise use their phones to access. This isn't anything new to QNX. We've been working with our partners to provide all the necessary elements to enable this integration through technologies such as HTML5, Qt, iPod out, MirrorLink, and Bluetooth. All these technologies can help create systems that minimize driver distraction but they represent only part of the solution. Pushing buttons on your head unit, combined with smart HMI design, does help, but it's not a panacea.

To truly help drivers keep their eyes on the road we have to minimize the time they spend looking at the infotainment display. Multi-modal HMIs built from the ground up with the assumption that high-quality speech recognition and text-to-speech are available will drastically change the way drivers interact with their infotainment systems. For instance, such HMIs could read your texts and emails aloud to you; they could even let you dictate responses at the appropriate time. But really, the possibilities are endless. And on the topic of talking to your car, we're constantly working with our partners to enrich the speech capabilities of the QNX CAR Platform. But more on that in an upcoming post.

By the way, I wasn't really using my smartphone while I was driving. That's illegal here. Not to mention incredibly dumb.

Controlled openness

Paul Leroux

The title of this post sounds like a contradiction in terms. But you know what? It captures the predicament faced by every automaker today.

On the one hand, automakers need to convince mobile developers that it's worthwhile to create apps for the car. They also need to make the process of creating and monetizing car apps as easy and open as possible. Otherwise, why would a developer spend time developing a car app when a phone app could reach many more customers? (About 60 million cars shipped in 2012, compared to more than 545 million smartphones — and most of those cars can't host apps.)

On the other hand, apps can't run willy-nilly in the car. For safety's sake, automakers must impose control on when specific apps can be used, and the apps themselves must be designed or modified to minimize distraction, possibly in accordance with government-mandated rules and guidelines. That may sound like an imposition on developers, but not really. After all, developers want to create apps that will prove popular with consumers, and consumers will be far more interested in apps that can be used while the car is moving — apps that, for safety reasons, can be used only when the car is stopped will hold less appeal.

But enough from me. Recently, my colleague Andy Gryc caught up with Thilo Koslowski, VP Distinguished Analyst at Gartner, and they discussed the notion of controlled openness for the car — along with how HTML5 fits into the picture. The cameras were rolling, so grab some popcorn, dim the lights, and check it out:

C3 recap: The future of the connected car

UPDATE: CE Week has uploaded audio and video of the C3 panels that Derek covers in this post. To hear what experts from companies like AT&T, BMW, Delphi, GM, and QNX see on the horizon for the connected car, visit the Connected Car Conference website — Ed.

Derek Kuhn

“Automotive has always been a wellspring of technology and innovation.” Those ten words, spoken by Doug Newcomb, car technology consultant and conference chair — and occasional QNX blog contributor — brought the Connected Car Conference (C3) to a successful close. The conference, co-located with CEA’s CE Week in New York City, featured panels on issues and trends for the connected car: big data, the future of radio, driver distraction, and more.

I was honored to sit on a panel that included executives from General Motors, AT&T Emerging Devices, and Audiovox, and that tackled the question on the minds of everyone in the industry: how can cars keep pace with consumer electronics? Traditionally, the speed of car development has trailed consumer devices, but with consumers looking at their cars as another connected gadget, the industry is working to bring technology into the car faster, while still providing a safe, reliable experience. As GM’s Tim Nixon put it, “we want to make the car better from the day you drive it off the lot.”

Striking a balance
Tim’s comment touches on something we frequently discuss — the significance of over-the-air (OTA) updates in ensuring that a car always has the latest technology. In fact, my colleague, Tina Jeffrey, just wrote a blog post on the topic; it's worth a read. Another point that came up is the need to balance security with consumers’ desire for cutting-edge technology. As I pointed out, not all infotainment systems are created equal — security shouldn’t be an afterthought in the pursuit of the latest and greatest tech. Rather, it should be deeply engrained in each step of the software development process. At the same time, consumer choice also has to be balanced with what OEMs are comfortable with.

Driving big data

John Quain of the NYT hosts the big data panel.
Photo: Doug Newcomb

John Quain of the New York Times hosted a panel on big data, which was full of insights on how data is being used to connect drivers and their cars. In response to the question, “how can big data in automotive save lives?” Delphi’s Doug Welk commented that, while data on crashes was abundant and readily available, data on near misses — which is even more important to understanding how to prevent accidents — is scant. Telenav’s Niall Berkey pointed out something that my colleague Andrew Poliak often discusses: the importance of the car as a sensor. For instance, by using information on how a driver is behaving, a car could activate assisted-driving technologies to reduce the likelihood of an accident.

Dealing with distraction
During the “Dealing with Driver Distraction” panel, representatives from the Auto Alliance of Automobile Manufacturers, Nuance, NVIDIA, and Pioneer spoke on how the industry is working to curb distraction. Gloria Bergquist of the Auto Alliance stated that the concern is nothing new; when car radios were first introduced in the middle of the last century, industry watchers claimed that drivers’ attention would be diverted by the novelty.

Gloria also drew from her organization’s recent report, which showed that most drivers overestimate how well they can handle distractions and think that it’s other drivers who can’t cope. Erik Clauson of Nuance discussed how voice recognition technologies — like the QNX intent framework — can play a large role in decreasing the cognitive load of drivers. Dave Anderson of NVIDIA defended skeumorphism — a design aesthetic that has received much criticism as of late — as a way to increase the intuitiveness of user interfaces and therefore decrease distraction. For example, digital instrument clusters that look like conventional (and familiar) analog instruments can enhance the driving experience.

Continuing the conversation
The day ended with a networking reception — a unique opportunity to pick the brains of the some of the industry’s thought leaders and observers. While I got to spend only a short time in New York for the event, I am look forward to next year when we can continue this conversation on the industry’s challenges and innovations.

UN agencies take major step towards international standards for driver distraction

June 27 marked a historic event in Geneva, Switzerland — an event that could ultimately lead to internationally harmonized vehicle regulations and Information and Communications Technology (ICT) standards to address driver distraction.

The event was a workshop titled "Intelligent transport systems in emerging markets — drivers for safe and sustainable growth". The title may sound innocuous, but don’t let that fool you. It only touches the surface of what was really going on.

So what, exactly, made this event so important? It was the first joint meeting of the United Nations (UN) agencies that deal with automotive regulations and ICT standards/radio spectrum allocation: the UNECE World Forum for Harmonization of Vehicle Regulations (UNECE WP.29) and the International Telecommunications Union (ITU), respectively. During the opening of the workshop, Eva Molnar (director, UNECE Transport Division) and Malcolm Johnson (director, ITU Telecommunication Standardization Bureau) spoke about the historic significance of this event and how they hoped it would be the beginning of a close collaboration.

This is big news. The possibility of vehicle regulations by the UNECE WP.29 may force automakers to work with the ITU, which has been working to develop comprehensive, internationally agreed standards to address driver distraction caused by mobile devices and other ICTs. Previous attempts such as the ITU-T Focus Group on Driver Distraction (FG Distraction) have had only limited success at engaging the automotive industry. See the FG Distraction reports for more information on the current state of such comprehensive standards.

Not if, but when
Regulation of ICTs could also occur. Strictly speaking, ITU-T Recommendations are non-binding, but they can become mandatory if referenced in a regulation by a national authority such as the FCC in the US. Increasing pressure to regulate use of ICTs in vehicles and the likely harmonization of ITU-T Recommendations with UNECE WP.29 vehicle regulations make regulations based on ITU-T Recommendations a real possibility.

Regulation of automotive and ICT equipment used by drivers isn’t a question of "if", but of "when". That said, many paths could lead to such regulation, some better than others. For example, authorities could jump the gun and issue regulations before good solutions are in place — and actually make the situation worse. With that in mind, let's hope that the step taken on June 27 is the first of many down a path that leads us to internationally harmonized standards and regulations that truly address unsafe driver interaction with ICTs.

Reducing driver distraction with ICTs

Inappropriate use of information and communication technologies (ICTs), especially mobile phones, is a chief culprit behind driver distraction and road accidents, and with automobile manufacturers scrambling to develop a “connected” driving experience, the ICT and automotive industries are becoming ever more closely entwined.

However, this integration of cars and ICTs need not come at the expense of driver safety, and there are strong grounds on which to argue that ICTs have great potential to enhance rather than diminish vehicle safety systems.

Under the banner of intelligent transport systems (ITS) the automotive and ICT communities are working towards a convergence of automobiles and ICTs that prioritizes drivers’ safety and broad consensus has it that international standards are the tools through which this will be achieved.

Over the past two years, as chairman of the ITU-T Focus Group on Driver Distraction, I have had the pleasure of leading a group tasked with laying the foundations for driver-distraction standardization work in ITU’s Telecommunication Standardization Sector (ITU-T).

Established in February 2011, the Focus Group reached the end of its study period in March 2013 and has been instrumental in raising awareness around ITU-T activity on driver distraction and the scale of this workload, as well as in providing clear direction to ITU-T’s driver-distraction work plan. The group has also been successful in opening lines of communication with key organizations and drawing new expertise into the ITU-T standardization process.

The Focus Group’s final deliverables take the form of five technical reports that describe:

• use cases and user interface requirements for automotive applications 
• system capabilities for improving the safety of driver interaction with applications and services (situational awareness management) 
• approaches that enable external applications to communicate with a vehicle

The reports are freely available here.

The conclusions put forward by the reports are being taken up by the two groups leading ITU-T’s standardization work on driver distraction, Study Group 12 (Performance, QoS and QoE) and Study Group 16 (Multimedia). New related work items calling for external coordination and collaboration may also be addressed by the Collaboration on ITS Communication Standards, a forum working to create an internationally harmonized set of ITS communication standards to enable the deployment of fully interoperable ITS products and services in the global marketplace.

Safe interaction with applications and services
The Focus Group’s work is just the beginning of an international standards effort to help drivers interact safely with applications and services — and not just apps on phones, but apps running in the cloud, in roadside infrastructure systems, and in the car itself, to name just a few locations.

The Focus Group’s Use Cases report details the use cases and user scenarios being targeted by this standards effort, but for now let’s look at Use Case 2, Scenario A (arbitration of external message), which illustrates how ITU-T is working towards a comprehensive framework for managing distraction and workload.

Keeping priorities straight
In this user scenario, a navigation maneuver is given priority over a social media ‘status update’ message. The blue call-out boxes indicate where the ITU-T Recommendations under development can enable safe interaction between the driver and applications. For instance, ITU-T Recommendation G.SAM will define mechanisms for prioritizing navigation, G.V2A will define the communications interface between the app and the driver-vehicle interface (DVI), and P.UIA will recommend characteristics of the auditory social media message.

Remember that the focus here is not on how to implement social media in the car, but rather on how best to manage workload and distraction.



Giving a navigation maneuver priority over a social media status update message

In for the long haul
Speaking from our perspective at QNX Software Systems, a subsidiary of BlackBerry, the work of the Focus Group marks the beginning of a long road ahead. Within ITU-T, QNX will continue to:

1. Work with the relevant parties to identify solutions to the problem of technology-related driver distraction and workload. These parties include automotive, telecommunications, and consumer electronics organizations; standards development groups; academia; and government agencies.
2. Determine which aspects of the solution should be standardized, and help drive this standardization.
3. Align QNX product roadmaps as solutions develop.

Certainly this is a long-term strategy that will take years to realize, factoring in the rigour of ITU-T’s standards process as well as the significant amount of time needed to deploy technologies in vehicles on a meaningful scale.

Join the discussion
A workshop hosted by ITU and UNECE at ITU headquarters in Geneva, 27 June 2013, will address “Intelligent transport systems in emerging markets – drivers for safe and sustainable growth” with a view to analyzing recent advances in ITS with emphasis on improving road safety in developing countries.

This workshop includes a session dedicated to driver distraction in which I will present the outcomes outlined by the Focus Group’s technical reports to spur discussion on the likely course of corresponding ITU-T standardization work.

The workshop is free of charge and open to all interested parties, including non-members of ITU, and online ‘remote participation’ will be available to all those unable to travel to Geneva. Please join us for what will certainly be a richly informative and interactive event!

This post originally appeared on the ITU Blog.

Enabling drivers to interact safely with applications and services

Since February 2011, QNX Software Systems has been leading an international standards effort to help drivers interact safely with applications and services. And not just apps on phones, but apps running in the cloud, in roadside infrastructure systems, in the car itself, and other locations.

If you jump to the end of this post, you’ll find a list of use cases being targeted by this effort. For now, let’s look at Use Case 2, Scenario A (arbitration of external message), which illustrates how we are working towards a comprehensive framework for managing distraction and workload.

Keeping priorities straight
In this user scenario, a navigation maneuver is given priority over a social media status update message. The blue call-out boxes indicate where the ITU-T recommendations under development can enable safe interaction between the driver and applications. For instance, ITU-T recommendation G.SAM will define mechanisms for prioritizing navigation, while G.V2A will define the communications interface between the app and the driver-vehicle interface (DVI), and P.UIA will recommend characteristics of the auditory social media message.

Remember that the focus here isn't on how to implement social media in the car, but rather, on how best to manage workload and distraction.



Giving a navigation maneuver priority over a social media status update message

Often, I am asked how this effort differs from the MirrorLink standard being developed by the Car Connectivity Consortium. The simple answer is that MirrorLink addresses only some of the use cases listed below. For instance, the scope of MirrorLink is limited to applications and services running on nomadic devices. Furthermore, adaptation of the driver-vehicle interface and external applications and services in the current MirrorLink solution uses a simple two-state approach, driving or not driving, which limits the ability of the vehicle to control the timing and modality of communications with the driver. Also, MirrorLink doesn’t adequately address arbitration or integration of communications with all external applications and services.

In for the long haul
At QNX Software Systems, our aim is to:

1. Work with the relevant parties to identify solutions to the problem of technology-related driver distraction and workload. These parties include automotive, telecommunications, and consumer electronics organizations; standards development groups; academia; and government agencies.
2. Determine which aspects of the solution should be standardized, then help drive this standardization.
3. Align QNX product roadmaps as solutions develop.

To be clear, this is a longer term strategy that will take years to realize. Both the standardization process and the time it takes to deploy technology in vehicles must be factored in. Therefore, we are also pursuing shorter term solutions, some of which I hope to cover in future posts.

The end of the beginning
The first major milestone in this effort was achieved at the closing plenary of the ITU-T Study Group 12 meeting, held on March 28 in Geneva. Here, the final report and 4 deliverables of the ITU-T Focus Group on Driver Distraction were approved. There was also approval of a liaison statement communicating these results to a large list of organizations working on this topic.

This marks the end of the focus group, but is really just the beginning for QNX and ITU-T efforts in this area. In future posts, I will explore various aspects of this comprehensive strategy.

---

Use cases and user scenarios targeted by ITU-T recommendations

Use Case 1: Interaction with external application/service
   a) Application on nomadic device
   b) Application on cloud-based server
   c) Downloaded Application
   d) Broadcast of roadway information
   e) Tethering
Use Case 2: Arbitration and integration of external message
   a) Arbitration of messages
   b) Integration of messages
   c) Both arbitration and integration of messages
   d) E-call
Use Case 3: Negotiation of network Quality of Service (QoS)
   a) Application selects network
   b) Application suspends interaction
   c) Application availability due to roaming
Use Case 4: Management of multiple dialogues
   a) Opening/closing an application
   b) Switching between applications
   c) Interaction with background application
Use Case 5: Adaptation of DVI (driver-vehicle interface) and external applications/services to driver abilities
   a) Driver with disability
   b) Dynamically changing driver capabilities
   c) Detection of impaired driver state
Use Case 6: Adaptation of DVI and external applications/services to roadway situation
   a) Driver busy notification
   b) Delay of message delivery in demanding driving situation
   c) Change message format based on road conditions
   d) Interruption of driver interaction
Use Case 7: Adaptation of DVI and external applications/services to vehicle status
   a) Vehicle enters safe operating condition (e.g., park gear, < 5 m.p.h., etc.)
   b) Driver adjusts vehicle controls (e.g., climate control, etc.)
   c) Suppression of hazard alert due to safe speed
Use Case 8: Adaptation of DVI and external applications/services to local regulations
   a) Application blocked
   b) Application suspended
   c) Interface modality disabled
   d) Age restriction
   e) Content restriction

For details on these use cases, download the FG Distraction Use Cases report.

Using smartphones to prevent traffic jams

Paul Leroux

Smartphones and driving don’t mix, right? Normally, you would get no argument from me. Driving is the one activity where a half-second lapse in attention can translate into a lifetime of regret.

But you know, there’s more than one way to use a smartphone in a car. Take Honda, for example. They’ve been experimenting with an approach that may help prevent accidents, rather than cause them.

Let’s rewind a bit. A few months ago, I wrote a post on the potential benefits of adaptive cruise control. These benefits include a dramatic reduction of traffic congestion and safer distances between cars.

Well, guess what: it seems that a smartphone app can have much the same effect. Recently, Honda equipped a number of drivers with an app that monitors acceleration and deceleration. When a subject drives in a way that avoids causing a traffic jam, the app screen turns green; otherwise, it shows blue.

Simple enough, right? And yet, the results were dramatic: formation of traffic jams was delayed by up 6 minutes and fuel efficiency shot up by 22%. Not bad for a smartphone app.

I am, of course, skipping a few details. Read more about the study in Tech-On!, an outlet of Nikkei Business Publications.

Autonomous cars? Suddenly, I’m not so skeptical

Guest post from Emil Dautovic, European automotive business development manager for QNX Software Systems

As a driving enthusiast, I have always felt a bit skeptical about the notion of autonomous cars. The reason is simple: I actually enjoy driving and don’t want someone else to do it for me, in this case the car itself.

Recently, however, my skepticism has begun to soften. I am fascinated, for example, by the SARTRE road train project, where a lead vehicle takes responsibility for a platoon of semi-autonomous cars. Also, recent research from the U.S. Highway Loss Data Institute suggests that, when it comes to some driving tasks, ADAS systems can already put many human drivers to shame.

Autonomous drive will become especially important when today’s “always on” generation starts to buy cars in earnest. They will, no doubt, want to consume multimedia and interact through social media even while on the road, and automakers will need to accommodate them.

HMIs with more (and less) distraction
What would this mean for car makers? Among other things, the infotainment system in a self-driving car could offer an HMI mode that gives the driver more freedom to pay attention to non-driving activities. When the car subsequently needs a human driver (for instance, it becomes disconnected from a road train), the infotainment system could disable these features and immediately go back to a less distracting user interface.

Also, driver assist systems — such as those for detecting animals and pedestrians — would need to be integrated with the road train system to decide how to react when, say, a rabbit runs in front of the car. For instance, should the car brake and warn other cars of the fact, or would it be safer to simply keep driving? It will be interesting to follow this initiative and see how the technical and business aspects evolve, and how, for example, the owner of the lead vehicle will be paid.

For another interesting example of research into autonomous drive, check out the BRAiVE project led by the VisLab team at the University of Parma. The BRAiVE project uses a variety of sensors, with a focus on low-cost alternatives that could realistically integrated into in production cars.

Bells and whistles
So what kind of impact could all this have on a company providing automotive software platforms?

There will, I believe, be an increased demand for a platform that could run all of these applications, enabling the advanced use cases while ensuring that critical functions always have enough processor power. And, of course, the platform will have to be reliable. If this same platform could offer all the bells and whistles available in consumer electronics and demanded by younger drivers, the self-driving future might prove to be a bit closer than we think.

By the way, if you’re unfamiliar with the SARTRE road train project, check out this video:

---

More about Emil
Emil Dautovic is an automotive business development manager at QNX Software Systems, where he is responsible for the European automotive market. Prior to joining QNX, he worked as a business area manager for The Astonishing Tribe (TAT), where he built TAT's automotive business from scratch and helped transform the company into an important player in the automotive HMI field with leading automotive OEMs and tier ones. He has also worked at AU-System (later Teleca and Obigo), where he served as a consultant on GSM base station development and as a sales representative serving mobile phone OEMs and ODMs worldwide. Emil holds an M.Sc. in Electronic Engineering from Lunds Tekniska Högskola.

Trend Spotting at SAE Convergence 2012

Guest post from automotive journalist Doug Newcomb

One of the Technical Sessions at the semi-annual SAE Convergence in Detroit on October 16 and 17 is titled Mega Trends and Their Effect on Automotive Electronics. While you’ll have to wait to find out what the participating executives, engineers, and analysts will reveal in the session concerning the rapidly evolving car technology space, here are three areas that are bound to be hot topics at the show.

Driver Distraction
This issue is at the forefront of everyone’s minds — automakers, suppliers, safety advocates, government officials, and consumers — as cars become increasingly connected. In order to help drivers keep their eyes on the road and hands on the wheel while still accessing the features they want, car companies and suppliers like QNX are developing cutting-edge technologies ranging from intuitive and configurable touchscreen displays to more accurate voice-activation systems that make control easier and less distracting.

Automakers are also being proactive in anticipating distractions: Ford is developing technology that assesses a driver’s workload so that some features can be deactivated in certain situations, and BMW’s pioneering work in “pupilometry” helps determine how drivers visually react when receiving information behind the wheel.



Ford's driver workload estimator (source Ford)

Standards
As more automakers integrate portable devices into the dash, drivers are increasingly frustrated by the fragmentation that’s occurring with first-generation systems. Features that are available for one smartphone platform may not be available for another, for example, and incompatibility issues are common. A push for an industry-wide standard has resulted in the Car Connectivity Consortium (CCC), of which QNX is a member. With MirrorLink, CCC’s industry-wide standard, portable device integration would be more straightforward and seamless for consumers. Getting all parties onboard will take significant effort though, since automakers have traditionally developed proprietary systems. But MirrorLink has substantial support, and the HomeLink system that’s allowed integration of garage-door openers into vehicles for years shows that such standards can be achieved.

Autonomous Cars
Two years ago, self-driving cars would have seemed like a distant sci-fi dream. But since the last SAE Convergence in 2010, Google has logged more than a quarter of a million miles with its fleet of self-driving Toyota Prius and Lexus RX450h vehicles. And this year the company has been instrumental in pushing through legislation that’s made self-driving cars legal in Nevada and California.

Audi is another pioneer in the space, developing an autonomous TT that drove solo up Colorado’s Pikes Peak. BWM has also debuted self-driving technology, and Cadillac recently revealed that its semi-autonomous Super Cruise lane-keeping technology will be available by the middle of the decade. Plus, Google’s announcement of its intention at the SAE World Congress in April to work directly with automakers and suppliers on self-driving technology will undoubtedly help accelerate this game-changing trend.

These are three topics are sure to be heavily discussed — and debated — at SAE Convergence 2012. Stop by the QNX booth during the show to see what the company is doing in these and other areas — or to share what trends you’ve spotted.

---

More about Doug
A widely respected reporter and editor with nearly three decades of experience in automotive journalism, Doug Newcomb currently writes for WIRED Autopia and for his own car technology portal, dougnewcomb.com. In 2008, he joined Edmunds.com as a senior editor, where he created the site’s Car Technology section. Prior to Edmunds, he worked as an editor for a variety of automotive publications, including Car Audio and Electronics, Car Stereo Review, and Road&Track Road Gear; he also contributed to many others, including Popular Mechanics, MSN Autos, Corvette Quarterly, and SEMA News. In 2008, he published his first book, Car Audio for Dummies (Wiley).

MirrorLink misunderstood: 8 myths that need busting

If you're new to MirrorLink, it's a technology that bridges the mobile phone and the car. It allows specially written apps running on the phone to be displayed on the car's head unit, where the user can interact with them.

MirrorLink is intended to extend the life of in-vehicle systems by allowing them to interact with mobile content and to support new features that didn’t exist when the car rolled off the assembly line.

Here's an illustration of how it works:


MirrorLink in-car communication. The protocol between the head unit and the phone can run over several transports, including USB, Bluetooth, or Wi-Fi. This example assumes Bluetooth for the audio back-channel.

When I talk to people in the automotive and mobile industries, I find they share a number of common misconceptions about MirrorLink, which I’d like to clear up. So let's get started, shall we?

1. MirrorLink is an Android technology. In fact, MirrorLink works with multiple mobile platforms. Phones using Android can support it, but so can phones from any other phone maker that supports the standard. Even Apple phones could support it, though Apple has currently chosen to go their own route with Apple-specific solutions.
   
   
2. MirrorLink allows any mobile app to run in the car. This is incorrect. A MirrorLink app can run in the car only if the car maker grants “trust” to that app. Each car maker has a different concept of what brands to promote, what features are safe, or what works well with each car. So, in reality, each app will be enabled depending on the individual make — or even model — of car.
   
   
3. MirrorLink promotes “driver distracting” apps. Also incorrect. MirrorLink is an enabling technology that doesn’t promote any type of app in particular. In fact, because the car maker must grant trust to an app, the app developer can't control what apps run in the car. That responsibility remains the domain of car makers, who tend to avoid anything that will cause distraction when displayed on a front-seat screen.
   
   
4. MirrorLink is the only way to connect an app to the car. There are in fact two others: iPod Out and HTML5. Apple supports iPod Out for Apple devices, which allows selected applications to output analog video to the head unit. (Note that the new iPhone 5 doesn’t support iPod Out.) HTML5 also allows mobile apps to run in the head unit, though its use in car-to-phone bridging is still in the early stages. QNX Software Systems has demonstrated concept vehicles that use BlackBerry Bridge (an HTML5-based technology) to connect an HTML5 app on a BlackBerry phone to the car’s head unit.
   
   
5. Mobile app makers will benefit most from MirrorLink. In fact, car makers may end up taking best advantage of the technology. That’s because they can use MirrorLink to customize and create apps, and to refresh those apps as a way of delivering fresh, new functions to their customers. MirrorLink gives them the ability to do this using a standardized protocol supported by most mobile platforms. Car makers could use MirrorLink very effectively, even if they never allowed any third party apps into their cars.
   
   
6. HTML5 and MirrorLink are incompatible. Not necessarily true. Current versions of MirrorLink use the VNC protocol to exchange graphical data. None of the advantages of HTML5 would be incompatible with a future version of MirrorLink; in fact, some members of the Connected Car Consortium (CCC), including QNX Software Systems, would likely be interested in merging these two standards. That would result in a new version of MirrorLink that uses HTML5 as the underlying communication protocol. (The MirrorLink specification is controlled by the Car Connectivity Consortium, of which QNX is a member.)
   
   Even if MirrorLink does go to HTML5, the industry would still need a VNC-based form of MirrorLink. VNC has much lighter requirements on the head-unit side, so it makes more sense than HTML5 if the car doesn’t have a high-powered CPU or lots of memory. The broadest possible option would be to have phone apps support multiple versions of MirrorLink (today's version with VNC plus a future version with HTML5) and to use whichever one makes sense, depending on what the car supports.
   
   
7. MirrorLink obviates the need for car-downloadable apps. Yes, MirrorLink capability is somewhat similar in purpose to downloading apps into the car; they both extend the functionality of the car after it leaves the factory. Because the customer’s phone will almost certainly be newer than the car’s electronics, it will have a faster CPU, giving the raw speed advantage to a MirrorLink app on the mobile. The MirrorLink app will also have guaranteed data access since the hosting phone will always have a data pipe — something that isn't certain on the car side of the equation.
   
   On the other hand, MirrorLink doesn’t give an app access to car features that would available to a car-downloaded app — features such as vehicle bus access, telematics features, or the navigation system. Also, a car-downloaded app would likely have a faster HMI than any off-board app, even if the mobile had a faster CPU, because of latencies inherent to screen replication. The car-downloaded app would also have better visual integration, as it could take full advantage of the car features, instead of appearing as a bolt-on product. Other factors, based on automaker control, compatibility, or product roadmaps could also favor an in-car solution. Even if you could address some of these issues, there would still be enough reasons for MirrorLink and an auto app store to live side-by-side.
   
   
8. MirrorLink apps can be built today. This is technically true. But, in their enthusiasm, new converts can sometimes forget that cars need to support MirrorLink for anything to actually work. Currently, only aftermarket car stereos support MirrorLink; no production vehicles support it. So if you’re a mobile app developer, the market for MirrorLink apps today is negligible. But expect this situation to improve dramatically over the next two to three years as production vehicles start to ship with this capability built-in.

Will adaptive cruise control spell the end of traffic jams?

Did you know that rear-end collisions account for about 30% of car crashes? For that reason alone, widespread adoption of adaptive cruise control (ACC) can’t come too soon. ACC helps prevent such collisions in two ways: 1) by maintaining a safe, preset distance from the car ahead; and 2) by applying the brakes quickly if that car comes to a sudden stop — more quickly, in fact, than most humans.

Good news is, ACC may soon become pervasive. The folks at Global Industry Analysts crunched some numbers and determined that annual installations of ACC systems will reach 6.9 million units by 2017.

Mind you, ACC isn’t just about safety; it’s also about traffic flow. For instance, a study by Suzuki and Nakatsuji (2003) suggests that travel times shrink significantly when at least 20% of vehicles on the road use ACC. And a study by Kesting et al. (2008) suggests that, in some scenarios, traffic congestion simply disappears when 25% of vehicles use ACC.

Example of adaptive cruise control
Source: Volvo

The picture isn’t all rosy, however. ACC may improve traffic flow, but not in every situation, such as merging from an on-ramp onto a freeway. That said, a study by L. C. Davis (2010) suggests that a technique called cooperative merging can significantly the improve the performance of ACC in this scenario. Meanwhile, a study by Jerath and Brennan (2010) suggests that the benefits associated with ACC may come at a possible cost — “self-organized” traffic jams. This effect, caused mostly by human behavior, may occur in a traffic system where most, but not all, cars use ACC.

Caveats aside, ACC systems continue to evolve. Some drivers tend to slam their brakes and use heavy throttle in traffic, creating congestive shockwaves that ripple down the highway. According to J.C. Power, newer versions of ACC help alleviate this problem by smoothly modulating brakes and throttle in stop-and-go traffic.

And now, a look at ACC from 1939…
If you think the concept of ACC is relatively new, think again. Over 70 years ago, GM created a “Futurama” exhibit for the 1939 World's Fair that showcased a scale-model highway in which cars automatically maintain a safe, efficient distance from one another.

GM predicted this technology would be in place by 1960. They got the timing wrong, but the idea right. Click the video to see a surprisingly prescient look at the car of the future — I’ve already bookmarked the spot for you:



What about you? Have you had much experience with ACC? And if so, has it helped or hindered your driving experience?

Am I crazy for talking to my car?

Earlier this afternoon, I participated in a connected car panel at SpeechTEK 2012, hosted by our friend Mazin Gilbert from AT&T. The other panelists included Greg Bielby of VoltDelta, Thomas Schalk of Agero, and Hakan Kostepen of Panasonic.

Even though Mazin did a fantastic job, not every panelist had a chance to answer every question. I was itching to answer some, so here are my responses to the questions that I didn't get to answer, or where I feel I could have provided a more complete response.

Have speech technologies matured to the point where they can be used robustly in the car? The general answer to this question from the panel was yes, but I think the real answer is a qualified yes. The technologies exist, but often aren't applied or may need auto-specific adaptations to handle in-cabin noise or other issues. Natural language recognition was an oft-stated driving technology, but a missing piece to the puzzle is hybrid recognition. I don't mean pushing recognition wholesale to the cloud, like Siri does. I mean a true split of the recognition effort, where each part does what it’s best at. Put the front half of acoustic processing in the vehicle to clean up the audio and convert the waveform to frequency-domain data, then send the data to the cloud-based server. The cloud server can then parse and interpret the data, and send back the result.

Hybrid speech rec solves three problems at once: better audio signals (the car can improve audio specific to the in-cabin environment), better cost (frequency data is far more compressed than raw audio, so you pay less for data transfer), and better responsiveness (hybrid rec gives the server time to start working on the response while it's coming in instead of waiting for the whole utterance to finish before starting).

Is driver distraction a major business driver, or is it the "Siri effect"? Currently, the car industry seems to use driver distraction as a reason to push a lot of features into speech. Many of those uses are gimmicky. Personally, I don't care if I can set my climate control system with voice — why would I when I can simply turn a dial? I once had someone ask me about the feasibility of adding voice recognition commands for rolling down the windows. I asked him, "Yes, but wouldn't people just push the window button?"

We shouldn’t implement speech commands just because we can. They may have contributed to excitement in the early adopter crowd, but we're beyond that now. Mind you, there are some seriously useful ways to use voice. For instance, any time you need to pick from a huge number of choices, voice recognition is the natural way to go. Calling contacts ("Call Sarah Potter"), entering destinations ("Go to 3121 South Park Street"), or picking music ("Play Audioslave") are all much easier than using an HMI to enter the same information, and safer to boot. It just has to work consistently and accurately.

Will car makers see more speech moving to the cloud, or will it be a hybrid of cloud and embedded? I disagree with the majority of the panel on this one, and, I think, the majority of people in the industry. Most auto people believe a hybrid between embedded and cloud allows the best of both worlds — good recognition and updatability when connected, and consistent reliability when not. My colleague Andrew Poliak also champions this view with a memorable catch phrase: Zombie Apocalypse. That is, you still want the system to work, albeit partially, when the infrastructure isn't available.

But if you ask me, everyone is missing the point — theirs is a technology-centric point of view. Everyday customer acceptance of a particular technology is notoriously harsh: if it doesn't work well, it gets rejected out of hand. Good cloud solutions beat an embedded solution hands-down; they just need some improvements (see my hybrid bullet above). Once a customer experiences a good solution, they will become frustrated with one that performs poorly. In my opinion, it's better not to offer the service at all, than to try a graceful degradation of capability, because most customers won't understand or care. Spend the effort instead on making sure you always have an acceptable cloud connection — either through multiple redundant mechanisms or a car-based powerful antenna — and you'll be better off. Even when the car knows some data that the cloud doesn't (like a mobile's contact list or music selection), there's no need to handle that on the embedded side. The cloud recognition server is powerful enough to not require the data set a priori. And I think we can predict an eventual migration of phone data to cloud-based data (or cloud-synchronized data) that makes the car's knowledge either easily transferrable or less relevant.

Who makes money, and how, from voice-enabled agents or voice services? This was one of the best questions of the panel, because nobody really knows the exact model, but everybody agreed that customer tolerance is very low. The most likely candidate is ad-based revenue. This doesn't mean reading ads aloud to the driver, but rather, positively influencing search results for either active or temporary situation-based points of interest (POIs). Depending on how valuable the service is to the driver, there will still be an option for service-based payments and high-value apps.

Standards and building mobile apps — will it come? You need standards if you want to build an app platform that will promote application creation and adoption. That's what we're doing with the QNX CAR 2 application platform — creating a way for someone other than the car companies to join the ecosystem and to deploy their apps to the car in a controlled way. But don't forget, you need a standard way to deploy apps for the cloud half of the recognition, too.

To close, let me share two photos. One was taken outside the Marriott Marquis, the hotel hosting the conference just off of Times Square in NYC. The other is from our PR agency, Breakaway Communications. What do they have in common? Wooden water towers. Sorry, I couldn't help myself; I just love those things. They just look so quaint in a city full of glass and brick.

For safety’s sake, why don’t cars just disable phones?

With all the focus on driver distraction, this is a question that I get asked occasionally. It’s a simple question, with a less than simple answer.

Using technology to control inappropriate phone use has been a topic at some of the driver distraction meetings I've attended. One proposed solution involves a technique called micro location — using ultrasonic waves to identify where in the cabin the phone is located. There are other ways to triangulate the phone's position, but they all require coordination between the phone and car. Knowing where the phone resides in the car is a requirement, as most passengers wouldn’t be happy to have their phone automatically disabled, just because they’re in the car. And the solution can’t be based only on the GPS speed of the phone, or you’d have lots of irate bus, taxi, train, or subway riders.

The fact is, unless all phone makers and car makers agree on the same standard, there's no incentive for either side to build half of a feature. You’d need to deploy potentially expensive technology that wouldn’t work unless you pair exactly the right phone with the right car. This likely won't happen unless companies are legislated to do so.

Given the speed of automotive development, it’s impossible for the car guys to build a technology that the phone guys won't leave in the dust, unless some guarantees are put in place. The adoption of Bluetooth is a good example. It took years before Bluetooth became widespread in phones, but its adoption had more to do with Bluetooth earpieces, not connections to cars. Car makers took a long time to roll out Bluetooth support as a standard feature because too many phones either didn't have it or had an implementation that wasn't fully compatible. Eventually, the two markets synchronized, but it took several years.

One argument against a technology-mandated disable is that not all jurisdictions agree on what is, or isn’t, allowable. In the US, 45 out of 50 states have some form of prohibition against using phones in cars. But what is disallowed varies widely by state — some don't allow any use of the phone (even hands-free), some prohibit teenagers but no other age groups, some disallow texting but not hands-free, some disallow use for commercial vehicles but not private vehicles, and some allow everything.

Another argument against a technological solution is that people can be educated to assume responsibility for their behavior. For example, why don't all cars have a blood alcohol level blow-tester hooked up to the ignition? Technically it's possible, but it's very expensive to do it from the car maker's standpoint. One could argue that it is worth it to have cars protect us from ourselves. But as a society, we've decided that, in the case of drunk driving, we are willing to give people back the responsibility. Rather than control the problem with technology, we socialize and educate people that driving intoxicated is an undesirable behavior.

We could, of course, decide to do the same with mobile technology, by educating personally instead of solving technically. This approach may make more sense than a technology-based prohibition: technology always moves at light speed compared to legislative mechanisms of control.
 

Report from CTIA Wireless: Apps in the Car

You wouldn’t think that CTIA Wireless, a mobile show, would be a good venue for a car guy. But automotive journalist Doug Newcomb put together a set of panels that managed to attract everyone from the automotive industry who attended the show.

I met a good number of friends from a variety of automakers, tier one suppliers, and hardware and software vendors. I also had the distinct pleasure of participating in one of Doug's panels, which was moderated by Damon Lavrinc of WIRED.

The topic was the future of apps in the car, and it generated a spirited discussion. Panel participants included Geoff Snyder from Pandora, Michelle Avary from Toyota, Henry Bzeih from Kia, and Scott Burnell from Ford — all experts on the topic.

Andy speaking on the

apps panel. Videos of all

the panels are now online.

In general, we agreed: apps are coming to the car. They have already arrived in several cases, and it’s only a matter of time before they come to mass-market vehicles. And apps are not for North American alone: it's a worldwide phenomenon.

Mind you, we engaged in lively debate on a number of questions: What role does the mobile app developer play? How to deal with the fragmentation caused by different OEM app platforms? How to deal with driver distraction? And when will the "one man app" ever make it into the car? We all had good and varied opinions on these topics, and the session was very well received by the audience.

Derek Kuhn, QNX vice president of sales and marketing, also participated in a panel session, titled "Can we all just get along… for the consumer's sake?". That panel focused on how the industry as a whole can create a more seamless experience for the consumer. Derek's co-panelists included Mark Harland from GM, Leo McCloskey from Airbiquity, Brian Radloff from Nuance, and Niall Berkery from Telenav.

Did I mention? Videos of all the panels are now on Doug Newcomb's website — check them out!
 

Find me a Starbucks! QNX concept car showcases power of WATSON speech engine

Yes, you can talk to the QNX concept car and tell it what to do.

Recently, our friends at AT&T invited us to bring the concept car to their "Living the Networked Life" event in New York. We said yes, of course! After all, what could be cooler than riding the streets of Gotham City in a digitally pimped-out Porsche 911?

Kidding aside, the event provided an excellent opportunity to demonstrate how the car takes advantage of WATSON, AT&T's natural-language speech engine. To get an idea of what WATSON can do, check out this video from Terrence O'Brien of Engadget:



For the full Engadget article, click here. And stay tuned for more updates from the Living the Networked Life event.