One of my favourite tasks in the OmAD newsletter has always been to report on Swedish research and development on automated driving and related areas. It is exciting to see that this much work is being done “locally”, something that you might not notice from general reporting in media. Thank you for all contributions!

As usually, it is a mixture of different topics that are in focus, from sensor development to user studies and measurement methods. I have, however, noticed that very few of the contributions are submitted by female researchers – do we really have this few women active in the field, or is it just a coincidence?

Why more training data cannot make up for poor annotations

A popular belief is that if there are annotation errors in the training data, it is often taken for granted that adding more training data will cover them up. To research this claim, Annotell did an experiment where they deliberately rigged the annotation of 2D object detection data to obtain low-quality annotations.

Annotell saw that the annotation errors in the low-quality data were not random but systematic and, instead of being “generalized away”, they were learned by the state-of-the-art object detector as if they were part of the annotation guideline.

One of the important conclusions from the study is therefore to always use high-quality annotations when training safety-critical perception systems, and never rely on compensating low-quality annotations with higher data volumes. 

The full article can be found here. For more information contact Oscar Petersson at Annotell (oscar@annotell.com). 

SMART

The aim of the SMART project is to enhance and further develop current state-of-the-art traffic models in order to enable analysis of future traffic systems. The project consists of two PhD projects, one focusing on microscopic traffic simulation and the behavior of and interaction between conventional and automated vehicles (Ivan Postigo), and one focusing on mesoscopic simulation and fleets of automated vehicles as a part of the public transport system (David Leffler).

The project is carried out by VTI, KTH and Linköping University and is funded by Trafikverket via Centre for Traffic Research (CTR). Contact Johan Olstam (johan.olstam@vti.se) or Wilco Burghout (wilco@kth.se) for more information.

First aid for AVs

Emergencies are bound to occur when autonomous vehicles (AVs) are deployed in real complex urban environments, but little research has explored how to recover afterwards, from crime to natural disasters. 

In our paper titled Autonomous Vehicles Could Help in Emergencies, we propose a speculative scenario of using a robot inside an AV to conduct first aid, exploring how to detect emergencies, and examine and help victims, as well as lessons learned in prototyping, especially related to perception. Positive preliminary results suggest the promise of robotic first aid in AVs in the future, in the aim to stimulate discussion. 

Paper: Martin Cooney, Felipe Valle and Alexey Vinel. Robot First Aid: Autonomous Vehicles Could Help in Emergencies. 33rd annual workshop of the Swedish Artificial Intelligence Society (SAIS 2021). June 14, Luleå. Funding obtained from KKS (”Safety of Connected Intelligent Vehicles in Smart Cities – SafeSmart”), VINNOVA (”Emergency Vehicle Traffic Light Pre-emption in Cities – EPIC”) and the ELLIIT Strategic Research Network. For more information contact Martin Cooney at Halmstad University (martin.cooney@hh.se).

Social media mining for drivers

Previous work on driver monitoring has focused on the short time that a driver is inside a vehicle and risks missing important health problems such as loneliness, depression, and sleep-deprivation that increase the risk of accidents.

In our paper titled Lonely road: speculative challenges for a social media robot aimed to reduce driver loneliness, we propose a speculative scenario of providing continuous monitoring and care via a ”robot” that interacts with drivers on social media, exploring potential challenges and solutions. For example, to address how to generate appropriate robot activities and mitigate the risk of damage to the driver, a hybrid neuro-symbolic recognition strategy leveraging stereotypical and self-disclosed information is described. 

Paper: Felipe Valle, Alexander Galozy, Awais Ashfaq, Kobra Etminani, Alexey Vinel, Martin Cooney. Lonely road: speculative challenges for a social media robot aimed to reduce driver loneliness. MAISoN 2021 (6th International workshop on Mining Actionable Insights from Social Networks – Special Edition on Healthcare Social Analytics, June 7 2021. Funding obtained from KKS (”Safety of Connected Intelligent Vehicles in Smart Cities – SafeSmart”), VINNOVA (”Emergency Vehicle Traffic Light Pre-emption in Cities – EPIC”) and the ELLIIT Strategic Research Network. For more information contact Martin Cooney at Halmstad University (martin.cooney@hh.se). 

Teaching about AVs

Autonomous vehicles (AV) promise to revolutionize our world, but training the next generation of researchers is a challenging task that traditional education strategies are ill-equipped to meet.

In our paper titled The CAR Approach: Creative Applied Research Experiences for Master’s Students in Autonomous Platooning we propose a new pedagogical approach targeted toward AVs called ”CAR” that combines Creativity theory, Applied demo-oriented learning, and Real world research context, reporting on its application to a master’s course with 10 small robots running ROS2 and Ubuntu on Raspberry Pi 4s in a cityscape mock-up.The results suggested the feasibility and usefulness of the CAR approach. Here is a video that explains CAR in more detail. 

Paper: Galina Sidorenko, Wojciech Mostowski, Alexey Vinel, Jeanette Sjöberg, Martin Cooney. The CAR Approach: Creative Applied Research Experiences for Master’s Students in Autonomous Platooning. In IEEE International Conference on Robot and Human Interactive Communication (RO-MAN) (pp. 899-905), 2021. Funding obtained from KKS (”Safety of Connected Intelligent Vehicles in Smart Cities – SafeSmart”), VINNOVA (”Emergency Vehicle Traffic Light Pre-emption in Cities – EPIC”) and the ELLIIT Strategic Research Network. For more information contact Martin Cooney at Halmstad University (martin.cooney@hh.se).

Automation and sleepiness

In our study titled Effects of partially automated driving on the development of driver sleepiness we compare the development of sleepiness during manual driving versus level 2 partially automated driving, when driving on a motorway in Sweden. The hypothesis was that partially automated driving will lead to higher levels of fatigue due to underload. Eighty-nine drivers were included in the study using a 2 × 2 design with the conditions manual versus partially automated driving and daytime (full sleep) versus night-time (sleep deprived).

The results showed that night-time driving led to markedly increased levels of sleepiness in terms of subjective sleepiness ratings, blink durations, PERCLOS, pupil diameter and heart rate. Partially automated driving led to slightly higher subjective sleepiness ratings, longer blink durations, decreased pupil diameter, slower heart rate, and higher EEG alpha and theta activity. However, elevated levels of sleepiness mainly arose from the night-time drives when the sleep pressure was high. During daytime, when the drivers were alert, partially automated driving had little or no detrimental effects on driver fatigue. Whether the negative effects of increased sleepiness during partially automated driving can be compensated by the positive effects of lateral and longitudinal driving support needs to be investigated in further studies.

Paper: Christer Ahlström, Raimondas Zemblys, Herman Jansson, Christian Forsberg, Johan Karlsson, Anna Anund. Effects of partially automated driving on the development of driver sleepiness. Accident Analysis and Prevention (2021). This work was supported by the ADAS&ME and the Mediator projects funded by the European Union’s Horizon 2020 research and innovation programme under grant agreements No 688900 and 814735, respectively, and by Autoliv Development and SmartEye AB. For more information contact Anna Anund at VTI (anna.anund@vti.se).

Shuttles in Linköping

Two autonomous shuttles are currently included in the test platform in Linköping, which is operated in collaboration between Akademiska Hus, Linköpings kommun, Linköpings universitet, RISE, Linköping Science Park, Transdev Sverige, VTI and Östgötatrafiken.

This test platform will soon be expanded to three shuttles. Also, an extended loop will be launched on September 15, 2021 in Vallastaden as a part of the EU-project SHOW. The goal is to enable first and last mile public transport for those who live in the nursing home and the children who go to Vallastaden School – the focus is on the users.

Furthermore, Linköping University, together with VTI, has been granted a WASP HS project called Autonomous Shuttles for ALL – AI, public transport, and people with disabilities, where people with special needs are the target group.

For more information about these activities visit the project website, or contact Anna Anund at VTI (anna.anund@vti.se).

AI-SEE

The Mercedes-coordinated project AI-SEE (Artificial Intelligence enhancing vehicle vision in low visibility conditions) started June 1st, 2021 as a part of Penta EURIPIDES². The project budget is 20 million Euro and will run for 3 years. The project will improve AI-supported sensor systems and supportive test and simulation methods with the goal of enabling automated driving in varying weather conditions.

Swedish partners in the project are AstaZero and Veoneer. For further information contact Jan-Erik Källhammer at Veoneer (jan-erik.kallhammer@veoneer.com) or Fredrik Åkeson at AstaZero (fredrik.akeson@astazero.com). 

CONVICTION

The ongoing FFI-project CONVICTION focuses on lidar and radar. It will be completed next spring. Project partners are RISE, VCC, ESI Nordic, Uniquesec, Veoneer and AstaZero. The project compares sensor data against simulation models, runs hardware-in-the-loop with radar, compares soft targets for test track against real models and develops measurement methods.

For more information about the project, contact Martin Sanfridson at RISE (martin.sanfridson@ri.se).

Iterative development for connected, automated vehicles

The goal of the FFI-funded project SALIENCE4CAV, which started in January 2021, is to develop methods that facilitate safety when working with agile development and frequent updates of safety-critical ADAS and ADS functionality in connected vehicles.

The project explores how to develop a safety case that can be maintained and updated continuously, how to handle variation in the safety assurance, how to design an architecture that facilitates frequent updates, how to handle aspects such as the use of machine learning and interaction with drivers, and how to effectively use field data to support development work. Results will be published continuously on the project website.

Project partners are Agreat, Comentor, Epiroc, KTH, Qamcom, RISE (coordinator), Semcon, Veoneer and Zenseact. For more information contact Fredrik Warg at RISE (fredrik.warg@ri.se).

SHARPEN – Scalable Highly Automated Vehicles with Robust Perception 

The global automotive industry is rapidly adapting to Deep Learning (DL) as a key technology, especially within perception for autonomous driving. This poses an increasing threat to the Swedish automotive industry. It is therefore of critical importance that the Swedish automotive industry builds world-class DL competence and excels in adapting the technology. The SHARPEN project addresses this need by developing novel concepts for Scalable and Robust perception systems, in particular within the Confined Area automation through Cost-Effective and Realistic Data generation. These concepts can be reused by the Swedish automotive industry, and the outputs of the project, such as the developed Deep Neural Networks (DNNs) and data generation tools, will also be utilized in future projects and products of the partners. 

Swedish automotive OEMs have an increasing need for robust perception modules to build a world model in autonomous driving applications. State-of-the-art research has mostly focused on perception in the public road setting with clear day conditions. However, the research on tougher conditions like night operations, adverse weather, sensor dirt presence, and confined area domain is very limited. Thus, SHARPEN will go beyond state-of-the-art and develop optimized real-time DNNs for 3D object and free-space detection working in these tougher conditions, which are the core building blocks of the perception system for confined area applications. These core blocks will be developed using sensor fusion. Furthermore, sensor failures during the operation of automated vehicles will be compensated for by applying novel SHARPEN sensor dropout and sensor-to-sensor mapping mechanisms which will be integrated into the sensor fusion. These DNNs will perform during night and day and changing weather conditions such as sunny, rainy, and foggy in a confined area.

The SHARPEN-project is funded by VINNOVA. Involved partners are EmbeDL, Halmstad University, Machine Intelligence Sweden AB. For more information please contact Eren Aksoy from Halmstad University (eren.aksoy@hh.se). 

Interactions with platoons

It is currently unknown how automated vehicle platoons will be perceived by other road users in their vicinity. Our study titled Safety and experience of other drivers while interacting with automated vehicle platoons explores how drivers of manually operated passenger cars interact with automated passenger car platoons while merging onto a highway, and how different inter-vehicular gaps between the platooning vehicles affect their experience and safety. The study was conducted in a driving simulator and involved 16 drivers of manually operated cars.

Our results show that the drivers found the interactions mentally demanding, unsafe, and uncomfortable. They commonly expected that the platoon would adapt its behavior to accommodate a smooth merge. They also expressed a need for additional information about the platoon to easier anticipate its behavior and avoid cutting-in. This was, however, affected by the gap size; larger gaps (30 and 42.5 m) yielded better experience, more frequent cut-ins, and less crashes than the shorter gaps (15 and 22.5 m). A conclusion is that a short gap as well as external human–machine interfaces (eHMI) might be used to communicate the platoon’s intent to “stay together”, which in turn might prevent drivers from cutting-in. On the contrary, if the goal is to facilitate frequent, safe, and pleasant cut-ins, gaps larger than 22.5 m may be suitable. To thoroughly inform such design trade-offs, we urge for more research on this topic. 

Paper: Maytheewat Aramrattan, Azra Habibovic, Cristofer Englund. Safety and experience of other drivers while interacting with automated vehicle platoons. Transportation Research Interdisciplinary Perspectives, 2021. For more information contact Maytheewat Aramrattan at VTI (maytheewat.aramrattana@vti.se).

First encounter effects in testing

To explore driver behavior in highly automated vehicles (HAVs), independent researchers are mainly conducting short experiments. This limits the ability to explore drivers’ behavioral changes over time, which is crucial when research has the intention to reveal human behavior beyond the first-time use. 

Our paper titled First encounter effects in testing of highly automated vehicles during two experimental occasions – The need for recurrent testing shows the methodological importance of repeated testing in experience and behavior related studies of HAVs. The study combined quantitative and qualitative data to capture effects of repeated interaction between drivers and HAVs. Each driver (n = 8) participated in the experiment on two different occasions (ca 90 minutes) with one-week interval. On both occasions, the drivers traveled approximately 40 km on a rural road at AstaZero proving grounds in Sweden and encountered various traffic situations. The participants could use automated driving (SAE level 4) or choose to drive manually. Examples of data collected include gaze behavior, perceived safety, as well as interviews and questionnaires capturing general impressions, trust and acceptance.

The analysis shows that habituation effects were attenuated over time. The drivers went from being exhilarated on the first occasion, to a more neutral behavior on the second occasion. Furthermore, there were smaller variations in drivers’ self-assessed perceived safety on the second occasion, and drivers were faster to engage in non-driving related activities and become relaxed (e.g., they spent more time glancing off road and could focus more on non-driving related activities such as reading). These findings suggest that exposing drivers to HAVs on two (or more) successive occasions may provide more informative and realistic insights into driver behavior and experience as compared to only one occasion. Repeating an experiment on several occasions is of course a balance between the cost and added value, and future research should investigate in more detail which studies need to be repeated on several occasions and to what extent. 

Paper: Jonas Andersson, Azra Habibovic, Daban Rizgary. First encounter effects in testing of highly automated vehicles during two experimental occasions – The need for recurrent testing. Information Technology, 2021. For more information contact Jonas Andersson at RISE (jonas.andersson@ri.se).

Ascetism and the use of infrastructure for improving testing for traffic safety

A very interesting and fruitful collaboration has developed over the last year between Viscando, Zenseact and AstaZero. It stems from a wish to enable future vehicle testing using real-life traffic information for generating test scenarios. It all started with a six-months study on improved testing in challenging situations under the Krabat umbrella, partially funded by Vinnova through Drive Sweden. Next, Viscando and Zenseact went on to carry out a proof-of-concept demonstration of the Krabat study under MobilityXlab, partially funded by Vinnova. In parallel, the three companies were granted a Vinnova FFI project, to take this exciting technology further, the ASCETISM project, which is now ongoing. These three activities are described in more detail bellow. For further information contact Katarina Boustedt at AstaZero (katarina.boustedt@astazero.com).

Improved Testing of Self-Driving Vehicles in Challenging Traffic Situations

This feasibility study focused on the cases of traffic weaving and merging at a highway on-ramp. These cases are selected because they are critical cases, with major impact on both vehicle software and sensor set, influence the sentiment of passengers and other road users, as well as having significant logistical impact on traffic flow.

In the foreseeable future, spanning decades from now, connected autonomous vehicles and human driven vehicles need to coexist on our roads. Therefore, there must be further exploration into the specific traffic situations which are the most challenging to the autonomous vehicles. Among these are highway on-ramps, where vehicle drivers normally adapt to the traffic flow and the actions of other drivers. Another example providing difficult scenarios for the mixed traffic is roundabouts, for the same reasons.

In order to develop AD and ADAS systems, industry needs a good understanding of complex traffic situations. This can be achieved by recording actual traffic flows in a real field environment. The project focused on the lane merging situation and assumed that it is possible to use modern camera technology and roadside units to record actual traffic flows. The objective was to define requirements on how an arrangement of equipment should be installed, and what data to capture in order to provide traffic flow information which can be used for repeatable test cases that meet the current and anticipated needs of vehicle manufacturers, regulators such as NCAP, and other stakeholders. Key steps of the project were: 

• Establishing current and anticipated stakeholder needs related to virtual testing
• Defining requirements on a test system for lane merging that can capture data for the virtual testing system
• Preparing for a Proof-of-Concept demonstration, to be carried out in a possible full-scale continuation project

More information about this project can be found here.

Proof of Concept: Accelerated scenario data collection for AD verification

The purpose of this project was to improve and evaluate the capability of the Viscando’s intelligent roadside traffic measurement system OTUS3D and its embedded AI software to collect traffic scenario data for scenario-based autonomous driving verification. The project goals were to implement an accurate geometric representation of traffic objects in geo-based coordinate system, and to demonstrate the system’s capability to reach the accuracy required for autonomous driving applications. The main outputs from this project were:

• Implementation of an accurate three-dimensional bounding box representation of traffic objects in earth-based coordinate system
• Validation of accuracy of object position and geometry showing considerable improvements towards fulfilling autonomous driving requirements

Conclusions were drawn on the current fitness level and further improvements needed for the OTUS3D system with respect to autonomous driving application requirements.

More information about this project can be found here.

AutonomouS and Connected vehiclE Testing using Infrastructure Sensor Measurements (ASCETISM)

The goals of the Ascetism project are to: a) identify and deploy roadside sensor-based naturalistic data collection solution to collect a dataset for extraction of critical merging scenarios for AV development, b) model the behavior of human actors in merging situations, c) demonstrate the concept of verification and validation toolchain of AVs using extracted merging scenarios and behavior models, and d) identify the gaps in the currently available data, such as quality and quantity and required improvements in data collection systems to enable scenario-based verification of AVs.

The project is expected to lead to solutions for using output data from cameras placed in the traffic infrastructure to generate scenarios for testing primarily self-driving vehicles. In extension, it may be possible to use the camera information and scenarios when planning road structures, ramps, roundabouts, and the like, adapted both for automated and mixed traffic.

Scenarios and driver behavior models will be derived from recorded data and scrutinized for integration into simulation and test track testing. This project will attempt to answer the following research questions:

• How can naturalistic data from infrastructure sensors be used in scenario-based verification and validation of autonomous vehicles?
• How do data properties, such as location, quantity, accuracy, number of recorded interactions, influence quality of simulations?

More information about this project can be found here.

Technologies, developments and trends

The rapid evolution of automation and communication technologies has led to the emergence of connected automated vehicles (CAVs). They support or replace human drivers in some or in all traffic situations, and help vehicles coordinate their actions. Large-scale introduction of such vehicles is thus anticipated to bring many benefits to society, including improved safety, reduced congestion, lower emissions, higher productivity, and greater access to mobility.

Our book chapter titled Connected Automated Vehicles: Technologies, Developments and Trends firstly presents automated vehicles including their history, categorization and current development. It then discusses connected vehicles with key enabling technologies, and the support for CAV and connected automated driving. CAVs bring many benefits regarding traffic safety, efficiency, and sustainability. For realizing those benefits, many challenges exist and developments remain an intensive area. The chapter summarizes the key benefits of CAVs, together with the main challenges and development trends towards the future connected and automated mobility. 

Paper: Azra Habibovic, Lei Chen. Connected Automated Vehicles: Technologies, Developments and Trends. International Encyclopedia of Transportation, 2021. For more information contact Lei Chen at RISE (lei.chen@ri.se).