Rémi Arnaud: Chief software architect

- My name is Remi Arnaud and I've been working in real time graphics forever. I'm obviously French. And I moved in the bay area a long time ago. Silicon Graphics actually recruited me out of my job and I was working for a flight simulator company in France. And doing some software to do image generation for flash mission, and since then, I've been in the bay area, and worked for big companies and small companies. Very energetic place, the bay area, so, I guess I'm spoiled now, and this is where I am. I'm working with a very dynamic and interesting company called Starbreeze. They are quite an old game developer, about 15 years old. They actually are a public company based out of Stockholm. They have very popular games that they created, namely Payday 2 is one of their best successes. They have a concept where you have to work, four people need to work in collaboration to be able to go through the missions of the game, and which, at the time was a concept that they created and worked out very well. They have a huge community of followers, and really what they want to do is to tailor to that community and communicate with them, and surprise them with newer releases, and we really like to release new content all the time, more like monthly or even a faster release schedule, listening to the community, and like I said, they're trying to surprise them and get them excited. I'm a technologist and chief software architect for new company and I oversee the software, and especially the game engine parts that we are building. The company is growing, it's got studios in Paris, in Croatia, in L.A., and we have engineers in Russia and so forth. So there's a lot of things for me to oversee in trying to steer all the technology to go in the same direction so we can grow the company and try to go together to the same goal. So E3 of course is our main show. Today we add Seagraph, which is the maker of graphics, graphics software and hardware, but as for the game industry, (mumbles). That makes sense that we actually chose E3 to release something that surprised everybody, actually, because we kept that quiet, and what we released was the most high-end VR experience that exists today. Because we're a small company it came as a surprise to all the big players that are in the space. So we work with a very nice IP, The Walking Dead, and as the company's growing including the technology there, it was a really nice thing to do to put together a mode that has new content, new engine as our own VR hardware, VR display which is called now StarVR. That was a very nice thing to be able to get all the company, all the new people together with one project. And so the experience is very intense. You actually wake up in a hospital and you don't have legs anymore. You're stuck in a wheelchair and there's a couple of your friends try to save you from, make you escape the hospital that's actually infested by zombies. And so it's interesting there's somebody behind you that's pushing you through the corridors and somebody in front of you opening the space. And at one point in the demo the guy in the CG is actually grabbing a gun from the floor and then he comes to you and he gives you the gun in VR but in a real world we actually give you a gun at the same time. It's interesting we have no explanation to give to people where immediately want to shoot the zombies because it happens like after the first demo. So we already are kind of, they want to survive and they understand that they're gonna have to use that weapon to make them self. And so what is very interesting is that we didn't know that it gets so intense and we get so emotional to the demo that we not only use the weapon to shoot at the zombies but when we get close we try to stab them with the weapon. Unfortunately some of the zombies are crawlers and they are on the floor and they actually smash the weapon on the floor. We have some broken weapons during the demo. We thought it was all new technology and we thought we may had crashes, everything can happen with this new technology, but the main issue was actually broken guns. Yeah, it's pretty unique because we really wanted to see, I mean, the story is where we want to provide to the community the best possible experience. And we realize that it was difficult. I mean, VR is very nice. I've been working on VR for a long time. It's very nice. But in the real world you actually have a lot of peripheral vision and you see things, even if you're not, you can't really focus on the things, but you see things happening on your side and that actually is part of the experience. And so we wanted to have a system that enables very wide field of view. The difference, it's hard to explain, you have to experience it yourself. That's the thing with VR. But basically the difference is instead of looking through binoculars or through a camera you actually feel like you're outside. I mean, when you put the StarVR system on you it feels like you're going out somewhere, it's not entering some space. You're going out into some space. That really makes a big difference. Although on that we push the limit on the resolution on quality of the screens and motion tracking and so forth. But it's all linked with the experience because we create our own content, because we have our own engine, try to solve all technical issues and because we do our own VR hardware we have the entire chance. We can focus and solve issues one by one to provide the experience based on the content we want to produce. There's something called uncanny valley where you see something that's almost real but not quite real and then that actually the closer to real you get the more the user will compare to reality and will see the difference. If you do something that's very stylized, very far from reality then nobody will want to compare that with reality. But the closer you get to reality the more people see the difference. And so that's where actually it's some kind of ramp that the closer you get the harder it gets. And of course VR gets you, it's not necessarily, when we try to reproduce the real world we try to get the players into our, at a separate world that they can experience. But it has to be, a lot of things has to be perfect. So we don't get distracted by it and we can focus on the experience. In VR there are a lot of details and technologies that, again, using working on the screen on a typical game can do a lot of tricks and special effects and short cuts that don't distract because it's presented in the screen that as soon as you are in VR and you can look everywhere you want and have this sort of vision. None of the streaks have RP cable without getting the player distracted from the content. That means you need to do a lot more in the modeling and the details. So the way you render has to be closer to how it works in the real world rather than special screen effects and things like that. There are no screens in VR basically so you can't paint on top of the screen once you have your image. You have to have all your effects in the 3D world, and that opens a whole new set of issues to solve in the modeling and in the rendering. I believe that to get to a very good VR experience there's a lot of tools and a lot of method that needs to be revisited from the ground up, and just adding VR to an existing engine may not cut it for the quality of experience we need. I was working it was not called VR. We had mounted displays and there was elementary space for flight simulators and the idea was to enhance the pilot with better aware of looking at his surrounding than his own eyes. I mean, this is like augmented reality, virtual reality. So now the big difference is obviously the money invested by big companies deciding to invest a lot, and also the fact the devices can be produced at a price that is affordable by everybody. So those are the big differences. Mostly because of the mobile phone technology, so mobile devices have bigger and bigger screens and those screens are the same you're gonna find in the VR devices. So that's the big, big change. Not much has changed as far needs and applications and the goals for VR. But its availability to a lot more people is very key. That's the interesting part actually. The more something is available to a lot of people the more innovation will happen because you never know what people do with technology and what direction it's gonna go. What we know though is that if only restricted amount of people have access to the technology then the applications will be small space, wherever the interest is. And when it's available to a large population then you never know what's gonna happen and that's the exciting part. Yeah, it was a military project and we wanted to have, to project images in the eyes. But at the time we didn't have LCD screens or things like that. We had CRT tubes and it was able to make small ones. We had a system where two CRT tubes are mounted on the sides of a helmet, made for one, and with a bunch of wires coming from the back. And the CRT tubes could project mirrors that went in front so you can either make them opaque or transparent and you can actually add an image, project image in the eyes of the pilot. All the issues with the systems were that you have maybe a thousand volts behind your back for the CRT tube. And the entire thing was heavy, so heavy that actually we had some kind of spring system to mount it on the ceiling to support some of the weight of the helmet itself. You see, that was the image generation technology we had at the time and there was a lot of things have evolved a lot in weight and low voltage and things like that. But that was already the kind of experience we could do, limited in resolution but we could readily already put people in a virtual world that way. That's a very good question because I don't think anybody has an answer to that. It's a hard question for anybody to answer. We don't know. It will chance beings. It's difficult for me to, coming to work is really intense experience in games for the community we work with and we really enjoy that. We're not doing toys and games for children, we really target a mature audience. In that case we notice that it's very hard physically. It's very hard to stay in the kind of game or the kind of experience we have for more than 10, 15, 20 minutes. Even gamers that are already used to it and really want to experience VR, they physically get tired. In our case we have too many players. Every time you're looking left and right, there's a lot of tension, a lot of excitement. I don't how this is gonna play because I mean, you sit in a movie theater for two hours even if it's a movie or (mumbles), and you enjoy and that's fine, but I think you can do that in VR for that long. You probably have seen that on TV, the one where they talk about the history of the movie theater. They created some movies to show people what movies could be. And we had some of the movies, the first movies, the kind of little demos, one of the movie is like a train that comes and arrive next to the camera. And so when you're in the theater you see the train coming toward you, a big train. And we had a film with people in the room and everybody's leaving the theater when they see that train coming through the screen. Well nowadays people would not run off the theater when we see that coming to us. Actually we say, "Give me some 3D glasses "because I want to see the train in my face." See, things have evolved a lot. People will evolve with, the experience will evolve and the audience will evolve and the demand will move and what exactly will be delivered is gonna depend on this, on what people expect from the experience and how much we can enjoy, I guess. How do we rate VR experience? Is there any rating system? How intense is it? There's a lot to do in that space. When I was starting I was doing flight simulators and the military had money to pay for developing new technology because they wanted to, I mean, it's very important. You train a pilot, a pilot you need, you don't want to kill the pilot in training. It takes a long time to train a good pilot. So really important on that. You can get as close as possible to reality without risk for the pilot. But other than that domain I noticed that most of the innovation are actually done by the game industry in graphics, in real time graphics. All the high end cards first get used by game developers in social experience. And then once it gets stable and once it gets maybe the standards come in and it's more reliable then it gets spread to a lot of other domains, but they add to it other contents. So it needs to be safe, it needs to be secure, can it be hacked? It needs to run for a month without interruption, whatever those things are there's a lot of extra work to do to add to what the gaming technology does. And then once you have that then it can be used in a lot of other space like in medical. So you understand you do need to have something that's safe and stable and because you will not want to use anything less than that in medical experience. And then there's in between. In architecture and, of course, things like virtual visits and travel, virtual travel and so forth that there would be, of course, a lot of applications. But technologies like innovating and looking at pushing the edges, and I think the right space to be in VR to do that is in the game space. We like to, I mean, like I said, Starbreeze really want to cater and work with the communities. There's a huge community of fan with the working that interestingly looks like quite a large population is interested on zombies nowadays. So we actually wanted to be there and present to all the fans of the Walking Dead, what the VR technology and gaming we're putting together can bring and what to expect in the future. So we actually took all experience the VR or VR system and working the demonstration. And we put them in a van and we are following the, how do you call that, the events, the VR events spread around the country with TV show where the fan come and communicate with the population. So we are here and then talking a lot and people can come and experience the demo. What's interesting for us too is the fact that we have all kinds of people. I mean, it's some set of population that like zombies which is pretty large, by the way, they really come and they experience the demo. And so we have avid gamers or people who never touched a game in their life or people that play game and never experienced VR and so forth. And we have all kind of people that come and we can see the reaction and see if there are issues with our system and fine tune it. And we're really happy with the result. Nobody got ticked and everybody really enjoyed it maybe too much. It was really a big success for us. So it's a surprise that we actually do that, but that's already a very big success and we'll continue to do that. Right now I have a lot of things I have to learn. There's a lot of things going on with the new generation of graphics API. Namely you have DirectX 12 and Vulcan and Metal, used to be Mantle. There's a lot of things to redesign as far as the graphics engine too, which is interesting because it gets closer to what you would do in a console versus a mobile phone versus a PC. Even though we're gonna get some kind of API that gets closer to Metal they still know where to have the same performance on no power hardware and high power hardware, I mean, they're totally different things. Yeah, I need to learn a lot of stuff. I need to constantly learn about new technology that I'll develop in the web space. When you're in the technology space that's what you do. You learn all the time. You cannot say, "Okay, I know everything now. "I can just apply it." That never happens ever. Putting them together on experience like the Walking Dead you have to have a lot of kind of people. It's designing the experience thinking about what it will be, creating the models for it, making the animation, painting, lighting, and then all the technology to render it, all the technology, motion tracking. You need a little bit of everything. That's what so interesting in the game industry is there's not one person who can do it. You have to have a lot of people with a lot of experience in a lot of domain. So it really is open to any kind of, so you could be somebody that likes to do models and you'll be doing modeling, or somebody who likes animating, you do animations, somebody that likes technology and you want to write rendering engine or somebody that wants to do hardware and wants to put together screens and optical systems. It's open to pretty much whatever people are, you have to be really motivated in what you do and you want to push that. But as of the open you need to understand what the end goal is. I guess it's the difference between research and game is you're not trying to create new technology just for the technology. You have to deliver at one point. At the end of the day you have people that may know anything about technology or animation, they don't even know how it's made but they're gonna experience it, and those are the people who are gonna tell you if what you did is good or not. How do you think about technology in that way where it's not, you're not gonna be judged, I mean you will be judged by your peers about how good your technology is but that doesn't matter. If at the end of the day your experience is not good, you failed. Same thing for timing. If you have a game to be released at Christmas then if you miss Christmas then you fail because, and the old company could go under if you miss the timing. So release in time and make good the experiences are the things you really have to be open to. And steer whatever you do, animation. Especially on the artists, we always, and actually programmers too, we can always improve it. "Oh, I can make it better. "I can spend more time there. "I can make the animation more." But is it gonna really make a difference at the end? The experience is the most important part. Is it gonna be noticed? Are you gonna miss the milestone because you're spending too much time doing that? Those are the hard things to learn. I mean, you always think you know and then you don't know. That's why you need one thing. I mean, if you're very specialized in one domain there's gonna be something else that will change, could change the fundamental things you study with and will put everything into redoing it. So, I guess, yeah, that's why I learn. You always have to stay open and listen and look at what's going on and don't hesitate to redo things. I used to say programmers never, as a programmer, a program is never good enough until you write it completely from scratch three times. It's a kind of rule of thumb that seems to apply pretty well to all the things I've seen so far. There's a couple of things that I know. Talking to people and listening to, I'm cabled as the engineer, and unfortunately what engineers do is they just want to fix problems. But by listening to people and understanding, trying to listen to what their problems are I come up with ideas on how to solve a problem. It's funny because in a relationship between two people some people just talk about their problem, and they don't want the problem to be solved they just want to say, "This is my blah, blah, blah." My day was, that happened, that happened. And if you do that to an engineer he's gonna want to fix every one of those things one after one. And he come with answers and solutions those things but the other person doesn't care. For some reason they just want to say what happened. They don't want to fix things, it's the way they are. So as an engineer it can be very overwhelming because if somebody has a lot of things who says I have a hundred problems to solve and he don't where to start. So communication is important. Also the other thing I thought about is hat I sometimes just want to be alone. And just like there's so many things that you've been receiving and you need to reorder them and just think about all those things and then find a common denominator or find something, there's something that's gonna pop out of that will be the one thing, "Oh, I always had thing "actually in mind and I always wanted to do that "but I never realized it was the case." And then you can, I don't know, is that meditation? I don't do meditation really but being able to cut off and spend some time just on your thinking about trying to recover and think about what happened, there's a lot of ideas that come out of those.