Software Governance and Automobiles - Session 4b
Accelerating the Development of Autonomous Driving Systems, by Mike Milinkovich.
EBEN MOGLEN: The next speaker who wants to talk about machine tooling for making autonomous driving software is Mike Milinkovich, of the Eclipse Foundation. The Eclipse Foundation is another important part of the FOSS ecology. Increasingly possessed of all the Java that’s worth possessing and an awful lot of long lifecycle software of various kinds, which it, better than anybody else, seems to know how to take good care of.
MIKE MILINKOVICH: So it turns out that I’m mostly here because of a cut and paste error, because what I sent to Eben said autonomous driving systems, and it actually turns out that we talk about at Eclipse is automated driving systems because we’ll get into this a little more. But it turns out that– the Eclipse foundation is a U.S. not-for-profit. We’re actually the oldest 501(c)(6) in the open source world. And I think, actually now, I’m the longest serving executive director in open source, which is kind of hilarious. I guess the gray hairs are worth something.
But we’re a U.S. company. We run it from Canada. We keep all our data in Canada so we’re mostly GDPR compliant. And we are much further along than we would be otherwise. But it turns out that when it comes to a lot of the technology and a lot of stuff we do, we are entirely German-focused. I think we’re one of the very few U.S.-based open source organizations that actually has a German subsidiary. And so we do a lot of work in the German automotive space.
So just to put this in perspective: 350 projects, 270 corporate members, 1500 committers, and we have 30 professional staff. And so that’s the Eclipse Foundation.
But I want to start off by, whenever somebody gets up and talks, particularly in a context like this, it’s kind of like, okay, so what’s what’s your story? What interests you? How many people here know–people who were with me at lunch don’t get to answer this– how many people here know what that is?
This is a Therac-25 linear accelerator that gives people with cancer large doses of radiation. And this is actually– in 1987, this was actually the first documented case of a software bug that killed people. So that’s why it’s interesting. It turns out that in 1987. Oh, by the way, and I’m from Ottawa, Canada, and they were made in Ottawa, Canada. And in 1987, I got cancer, and I spent six weeks on one of these. And so, the fact that I’m alive is just basically a matter of either not operator error or just some good old-fashioned good luck.
And so for me when we talk about functional safety of software systems, it’s kind of ironic. I was actually studying for my masters in software engineering at exactly the time that this was going on. So it’s just an incredible coincidence of just amazing like, well luck or bad luck or good luck, but it was just a pretty amazing scenario.
And the other thing to know about me is I really, really, really like to drive. That’s actually me on a race track and yeah, I like to drive fast, and I’ve probably spent more hours studying driving than most autonomous driving systems have. But it’s good fun.
I actually added this slide today as we were talking because I thought about this as we were going through, really what a lot of what we’re talking about through this day today is about instilling trust in the systems that are about to increase. And we’re talking with autonomous driving and automotive in general, but this is just one of the many vectors in which computers are taking increasing control of our lives.
And really what we’re talking about here is trust across the lifecycle, and I think this morning, Mark’s talk about the SNAP’s and what he was working with with Ubuntu, that’s really about software provenance. How do you know what’s actually running on your machine? Software safety, really Nicholas’s talk and that was a great rant, man, I really loved that talk, is knowing, trying to know what the software is actually going to do. We’re being able to reverse engineer what the software is going to do.
But, one of the things we haven’t really talked about, and I’m sure that you’re really aware of this, but it just didn’t get to it with the time you had, but there’s, also now increasingly going to become important, is this concept of data provenance which is sort of this yet-untouched area of error that can enter into our systems. Because as more and more, it’s not about programming, it’s about actually, machine learning. The data that goes into the machine learning algorithms is what’s going to ultimately determine their behavior.
And actually, how many people heard this thing– so about a month and a half ago, I think, where Amazon Alexa’s were maniacally screaming and laughing in the middle of the night? Right? Like, I don’t have one. I don’t think I ever will. But, so that’s actually an example of data poisoning, or at least they think it’s an example of data poisoning, which is perfect, and this is almost a trivial example. Somebody doing a prank, but by putting in, understanding a little bit about how the algorithms works, putting in some bogus data, they actually got this really unhappy experience.
But imagine if you actually had systems that were doing something much more critical, that where we didn’t actually completely manage the provenance of the data and a nefarious actor was able to pull off something much more problematic.
So really, I think, what we need to do is we’re talking about growing these systems, and open source is definitely part of the solution here. If you can’t read the code, why would you ever trust it? And if you can’t share the code and learn from the code, again, why would you ever trust it? So open source is part of the solution, but we have so, so much more to do on so many other aspects before we can think that we’re done.
So I’m here to talk about OpenADx, and we’ll return to it a little bit. So, OpenADx is a project that we’re just kicking off at the Eclipse Foundation that has aspirations of creating an end-to-end toolchain. It’s really more of an end-to-end ecosystem for doing all of the tooling that you need for building automated driving systems. And so if you look at the toolchain that automotive companies are using today for what they call level two and level three systems. Sorry, I can’t remember what the name of it is. In automotive, there’s this five-level system where basically level 1 is what we know of today as cruise control. Level 5 would be fully autonomous driving. Right? And most automotive companies these days are working on level two and level three. They’re just kind of maybe getting started with level 4, but it’s not, it’s just they have a long way to go and they recognize they have a long way to go. But for the folks that are working on the level two and level three systems today, this is the kind of toolchain that they’re grappling with.
The thing to understand is that on every single one of these nodes in this linear system here, you have different vendors, different tools, different data standards. Some of them are proprietary, some of them are open source, and frankly it’s just a complete mess. And so what the OpenADx solution is trying to do or what we’re trying to accomplish, and this was primarily initiated by Bosch who is one of the companies that we work with quite a bit in Germany, and it’s really about trying to have a notion of industry-wide interoperability for tools for helping to build automated driving systems.
But before I can really talk about OpenADx, I have to talk about some of the other stuff that’s happening at Eclipse because without sort of the context of the other things that we’re building, it doesn’t really actually make a lot of sense, or it certainly wouldn’t be like, why would you be doing that at Eclipse?
So one of the other projects that we have is called OpenPASS which is specifically focused on doing simulation for being able to test automated driving systems. So OpenPASS stands for “open platform for the assessment of safety systems”. And, this is, don’t think of it as a single simulation engine. Think of it as a platform for combining many simulation engines to test many, many scenarios.
And so this is– the group of companies involved, the Volkswagen Group, and Audi is involved. And the initial code contribution came in from VW, if I’m not mistaken. But there is one symbol on there that’s actually particularly interesting, and it’s the most recent organization to join this little adventure, and that’s TÜV SÜD, which is actually the safety authority in Germany.
And actually, that’s one thing we haven’t really touched on enough today, I think, is like, we’re, you’re sitting in New York. In the U.S. The regulatory regimes in Europe are completely different than what we have here. And I think it’s fair to say that the Germans are particularly inflexible when it comes to modifying the rules just because the companies are jumping up and down. Maybe that’s a gross generalization, but the fact that you still can’t get an Uber in most of Germany I think is a reasonable example of that. They have their rules, and they stick to them. And that’s usually a good thing.
But the fact, the reason why TÜV SÜD joined this organization is because everybody is motivated to increase the level of safety, and the more that we can do that without having humans behind steering wheels, the better.
And so, I think right now the general rule is you have to drive a model of a car for 100,000 kilometers on German roads before you can have even a hope of getting it getting it qualified with a new system. And the goal of OpenPASS is explicitly to cut that number in half. So this is actually big money, right? If you have a human being behind the wheel for 100,000 kilometers, that’s expensive. Right? So if we can reduce that number through simulation, we’re going to make make progress faster.
So the kinds of things they do are accident simulation, traffic simulation scenario variation. So you can play more games with the sensors in the systems and see how it reacts. You can play more games and do that way faster through simulations than you can through real world.
Another project we have at Eclipse is Eclipse SUMO, and this is this is a really interesting, very, very large city simulation. So their goal is to basically model everything that’s moving in a city at once. And this comes from DLR which is the German equivalent of NASA. And what they can do… They’ve actually got to the point where they can actually do a pretty decent job of modeling all of the stuff moving in Berlin, which is a pretty decent sized city, so at once they can be simulating all the cars, all the bicycles. I think they have it right down to the bicycles. I don’t think they have it right down to the pedestrian, but all the trains and light rail and everything, all moving at once. So it’s been around for quite a while, and it’s pretty exhaustive so you can do it with or without graphical UI, and so basically what they need to model other other cities is just better data. But that’s another project we have at Eclipse.
And then a third one I just wanted to mention in passing is Eclipse Kuksa, which is partially funded by the European Union’s H2020 research program, and what it’s doing is building a complete open source platform for connected cars. So this isn’t just simply about autonomous driving or automated driving. This is just about one of the precursor technologies to completely enabling that, which is a full stack for doing connected cars.
So Ericsson and Bosche are the two primary industrial partners in this, but it’s basically tackling everything from the 5G layer up to how you aggregate and manage the time series data in the cloud and everything in between. So it’s a pretty ambitious project. Oh, and by the way, a full IDE for developing applications for the platform as well. So it’s pretty much an end-to-end solution, including– solution is definitely the wrong word. This is a research program in an open source project. It’s not ready for, and it just getting started. So solution is too strong of a word, but the vision is that this is a complete scenario, a complete stack if you will, for doing connected car, including the ability to have a full toolset for building software for it.
So that little bit of context, let’s go back and talk a little bit about OpenADx and what the goals are there. And so automated driving is a complex challenge. I think we probably have figured that out after being at this for almost eight hours. So there’s a lot of different aspects that go into it, and there’s a lot in building these systems.
Well, I think, Eben says it’s impossible. Others believe that it’s maybe within the realm of possibility within 40 years or so, but no matter where you are on the ambition scale, I think it’s fair to say that they’re complicated enough without having your tool set or the tools that you use to build the systems being a complete mess, which is pretty much the scenario right now.
So it is a very long chain of requirements to put together a system and test a system like this, and none of these tools were ever designed to work together. So, therefore, the idea is to use the open source techniques that we’ve had lots of practice at and see if we can come up with something that works better.
And there is value here for everybody involved in the toolchain. So whether you’re an OEM, or not the toolchain, in the supply chain. So whether you’re an OEM, automotive tier one, an IT or tech company, semiconductor company, or an engineering service provider, all of the various players in this automotive ecosystem, building these systems, have business motivations for wanting these kinds of tools to exist.
So as we’ve already talked about, the idea is to build this complete toolchain and use this as a reference architecture to ensure implementations, interoperability. And the way that we’re doing this, is because this is an integration story, it’s not just simply building a bunch of software and, saying that we’re going to do all of this in one project. This is a story of integration. And so really what the focus initially, and certainly for the next year or two, is primarily around test beds.
So basically pulling together tools that work in parts and demonstrating their interoperability and using them to build a solution for something that does more than what they can just do by themselves. So the glue code that’s actually going to be used to implement these integrations is all going to be done in open source. And the goal over time, we hope, is that this will be used to drive standardization. We have no particular ideas on where that standardization might take place, but that’s the goal eventually.
So first, two test beds that are currently under construction right now: one is focusing on simulation. So it’s actually using some of the pieces from OpenPASS that I was talking about earlier to actually demonstrate simulation for automated driving systems, so basically testing scenarios out and seeing how the automated driving systems react.
And then the other testbed that is under construction is around data storage, data management. So, as cars generate an enormous amount of telemetry data, as I’m sure you’re aware, and so capturing that data and being able to manage it and then being able to feed that back into the simulation and test engines.
So those are the first two, the first two test beds that are currently under construction.
So the timeline. So they just wrapped up a big conference in Berlin last month called Bosch Connected World. We’re part of– they had, as part of Bosch Connected World, they had the largest hackathon in Europe. There’s like 450 people there, whether from startups, students and a great, great mix of people and several of the groups that were in the hackathon were hacking on this particular problem and did some pretty interesting demonstrations, and then coming up pretty soon as a DDS-based simulation tool connector for– DDS, I don’t know if you, how many people know that it’s one of the IoT related protocols, and it’s actually one of the underpinnings for– I can’t remember who was mentioning ROSS, but it’s at the core of ROSS for how it distributes data through the systems.
So with that, thank you, and I think then we can throw it open to questions.
Previous: 4a-gilpin | Next: 5-autonomous-panel | Contents