24 Jun 2017

Note: I am now writing thing almost a year later so some of the finer details have been lost or are a bit fuzzy.

One of the nice things about having the competition in the state of New York was that we had a logistical advantage in that we just needed to drive down in a few hours to get to the state of New York. By renting a van, we could also bring all the equipment we needed including a power supply, a computer screen and a desktop. One of the guys from Autero screamed “A DESKTOP?!” when he saw us.

As I usually do when I go to competitions, we arrived a day early to attempt to scope out the site and get a better understanding of what was happening. Unfortunately for us, the airport is a legitimate airport with security and all and we couldn’t actually go in to inspect the site. The DJI employees were pretty nice to us and actually gave us some spare parts we needed to fix our GPS mast. You could feel they really wanted everyone to succeed.

As we found our assigned tables we set up our equipment and I was stoked at how well organized we all were. The logistical advantage was visible, we were the only team powering our quad using a power supply instead of batteries.

Once set up, I was keen on looking at other teams to see how they had modified their M100s to fit the mission needs. On the surface everything looked the same but it was the small details that were interesting. KAIST had what looked like a secondary stripped down camera glued to their x3, Autero had used custom made metal brackets to elevate their front and back facing guidance sensors and most teams had found a way of further elevating their GPS module to avoid interference coming from cables connecting the X3 to the manifold. One team only had two people, rumor has it the other team members had visa problems.

We sat down for the briefing and learned about how the competition would go down. As a matter of keeping good records, I even kept the presentation pdf. Fun fact: The night before, I walked in the hotel lobby and struck up a conversation with Robert Schlub who was working on this presentation on his laptop (more on him later). Even the DJI guys do some stuff at the last minute!

We were informed that we would get a single practice run followed by our official run the next day. We felt this was a bit ridiculous, a single practice run meant you needed everything functional before you actually got to Rome, NY. You just have to play the hand you’re dealt. I was stressed out to say the least. In fact, you could say I was pretty close to shitting my pants right there, I couldn’t stop pacing around near our table.

First practice run

During prep time I had a chance of speaking to Robert Schlub Vice president of R&D at DJI, who was one of the people running the competition. Little did I know that he used to be director of antenna design at Apple, so I looked pretty ridiculous coming in with my huge Ubiquity antenna. But he accepted the idea of me installing it in the truck for our use.

Our turn came up and we were shuttled from the hangar to the test site. We could finally see the search area. What. The. Fuck. They brought out a fucking Cadillac and put an AprilTag in it. How the fuck would anyone find that survivor. Most of the search area seemed pretty legit, from afar we couldn’t tell but there was actually a lot of space between all the objects, enough for a quad to easily navigate around. Our timer started and we leaped into action, everyone had a very specific task or thing to set up and we all knew what to do. I was in charge of setting up Wifi, and I had already all my PoE adapters and access points connected. I go to the truck and plug in. Nothing happens. I figure out I didn’t connect a cable right. Still, nothing. I see my PoE adapter’s light keeps flickering. Looks like I’m overloading the truck and the whole system shuts down. What the fuck man.

I tell the team what’s happening and we attempt to plug our setup at the HQ and cross our fingers to see if the range holds between HQ, the truck and the search area. Something still doesn’t work. Fuck fuck fuck. We try a bunch of things, Robert offers us one of his own PoE adapters, in case we had fried ours, we try piggybacking on DJI’s own network, still nothing (I think we might actually have misheard an IP or something). At some point time is running out but things seem to be working for us to attempt a run. Alex gets in the truck and goes to start the run. The truck starts moving and Alex presses the start mission button in the app… Nothing happens. We were the only team that day who didn’t even take off from the truck. We were beyond disappointed with ourselves.

You see, one of the problems of our system was that we didn’t realize we could configure a network adapter to be always on with a static IP (this is how it’s done on AscTec products). That meant that if our quad didn’t have Wifi, it didn’t have an IP address. As a result, ROS would fail to start. In general this wasn’t a problem because we had always had network connectivity whenever we tested. But the whole Wifi fiasco meant a core part of our architecture was missing.

We decided to stay in the hangar and decouple our system from the need for Wifi. Fortunately for us, Gabriel and his Linux skills were of great help. I think we ended up putting a static IP and setting all ROS scripts to point master to localhost instead of the Manifold’s hostname.

Practice run 2

We were relieved to learn that DJI would allow all teams a 2nd shorter practice run. Looked like the practice runs didn’t go so well for everyone. I heard cardinal SAR went straight into a tree.

We decided to use this practice run to attempt to go into the arena. As the truck starts moving, the M100 shoots up and goes over to the search area. Yes. The M100 lowers down into the area and lands. All good, we had done this to reset the internal altitude estimator. The M100 takes off and… gets stuck. It seemed like the place we decided to land was actually between an obstacle and a virtual wall. The M100 couldn’t move. In fact, this problem was something we were always unsure of. We didn’t know where in the search area we could safely descend. One of the things we had investigated was writing a grass detector and descend where there is grass, i.e. no obstacles. Another was to do a bit of mapping on a small area and try to descend somewhere far from obstacles. All of this was nice but we hadn’t had time to finish these modules and it turns out that most other teams were doing the same thing we were and simply picking a GPS coordinate and crossing their fingers. Except Autero who seemed to enter the arena while also running their obstacle avoidance module. This irritated me a bit because if you read the rules, the diagrams suggest that the search area would be surrounded by walls, which led us to believe we had to come in from the top - not the sides.

2nd practice run, wasted.

End of day 1

At the end of the day we were extremely disappointed in our performance and completely exhausted by the whole ordeal. Normally when I go to competitions I make sure to take a lot of pictures and document everything for the school’s communications department because they love to report on the things we do. This time was different, we were so stressed that we didn’t event have time to take any pictures other than things that would be useful for strategy later on. Fortunately, we met /u/rubiksman from Reddit at the competition and he even took a picture of our team in his imgur album. Hooray, there’s proof we were there! See image 34 below were we can see from left to right Quang, me (used to have a man bun), Alex and Tien. Gabriel was at the bathroom at that time unfortunately.

Dji Developer Challenge 2016

Once we left the airport, we took in a breather to gather our thoughts and figure out where to go from here. After a mandatory pizza break, we decided we would go all in on our moving landing system and bank on that to win the competition. No one had been able to land during practice day and the way points are setup, a moving landing would be worth more than actually finding all the survivors. Stephen Wu later confirmed that this was done on purpose.

We also received the following email around 18h48.

Dear Teams,

I hope you all had a bit of fun today and we’re hope you’re even more excited for tomorrow because there can only be ONE winner!

As announced after the competition today, the details are as follows:

• Safety - Please do not fly near the crowd. A couple teams (you know who you are) flew very close to the viewing area today; NUAIR was very lenient today but will not be when there are general public present. Any aircraft that gets too close will result in a safety mission abort.
• AprilTags changes. Today, the AprilTags were placed in different orientations; for tomorrow, we will have 4 AprilTag on the floor (or as close as possible to the floor) and one up against a wall.
• Format Change to something similar to 27th reruns.
  • For the final competition, teams will have 45 minutes to run as many tries as they like or can.
    • Teams will get a warning 10 minutes before the previous team is done.
    • If the team before them does not want to or is unable to complete the entire 45 minutes, the team will have 10 minutes to get ready before their 45 minutes begin counting down
    • Once the team’s time slot begins, 45minutes starts to count down
  • At the end of the 45minutes, the Car Judge will execute a manual landing and no points will be awarded for landing.
  • Each valid trial will be graded; a valid trial means the drone must take off from the moving vehicle.
  • The highest score within the 45 minutes will be counted.
  • Teams can now choose to either wait 2 minutes after take-off before landing or enter and exit the searching area before landing
  • Two teams must be at the tent on standby at all times
  • There will be a powered staging station for teams to prepare at the tent as close as possible to the car
• Truck speed will be lowered to 15mph unless the team requests to land on 20mph moving car

We look forward to seeing you all again tomorrow; try to get some rest tonight (who are we kidding, you’re all going to be hacking away!) and Good Luck tomorrow!

Best Regards, The DJI Dev Team

Awkwardly enough, we were one of the teams who had flown too close to the crowd. The way the trees are positioned, there was a clearing close to the crowd that could get you unobstructed to the search area. So multiple teams including ours setup GPS waypoints through that corridor and this made the organizers a bit uneasy.

At this point it’s clear we’re all exhausted and yet there’s a long night of hacking in front of us. We needed to:

• Make sure our quad could work without network connectivity.
• Make sure our quad could land in simulation.
• Make sure IMU and GPS data could be sent through the remote control instead of Wifi. The mobile SDK supports transparent data transmission for low bandwidth data. Its not great, but it could work for our use.
• Write some bash scripts to start all our nodes when the Manifold powers up.
• Drill the steps to reset and start the quad to attempt another landing.

Basically our strategy was to attempt as many landings as possible in the allotted time. Speed was of essence and any little trick such as start scripts or command aliases that could save us time in rebooting the system was essential. We decide to have our quad gun it to the search area and come back instead of incurring the 2 minute wait time incurred by staying close to the truck.

I was so tired that it took me almost an hour to set up the team in a conference room with full Wifi and ethernet connectivity. One bad idea after another, we thought turning on the router in our rooms would get us wifi at the ground floor. At some point we realize that the ethernet jacks in the conference room weren’t working and we asked the front desk clerk of all people to help us figure out how to get Internet working through those. The front desk clerk.

We setup Alex’s messenger bag with all of our charged batteries. We went through all the steps from battery on, to plugging in a laptop through ethernet to starting or resetting some nodes, setting down the quad and starting the mission on the mobile phone. We drilled it multiple times. It had to be second nature to Alex. At some point, I notice in the simulator that the quad is going upwards instead of downwards even though our vision pipeline is working and can see the AprilTag. I ask Alex about it and he figures its just DJI’s simulator being wonky. Big mistake. I take is answer for granted and move on. Did I mention we were tired and not thinking straight?

We call it a night and go to our rooms to hopefully catch 2 or 3 hours of sleep.

Day 2 - D day

While the team stays in the hangar to set things up I split off and go to the staging area to observe the other teams. I was hoping to get some strategic information and I figured I wouldn’t be of any use back at the hangar anyways.

After a few teams flew it was clear everyone had thought the same thing we did. The safest and probably only way of winning this competition was to be able to do a landing. Even a static one might win, since most teams who entered the search area actually failed to get out. So might as well take off and land without risking getting stuck in the search area. I don’t remember clearly if TU Graz was able to show their static landing again but I remember Cardinal SAR saying “screw these trees” since they had crashed in trees twice now and simply going for the static landing. One of the Chinese teams attempted a moving landing but kept missing the mark by about a foot.

When Autero flew, things got a bit more tense, their strategy was to attempt multiple moving landings. They succeeded once and then attempted a landing with a search for survivors. If they could find at least one survivor and land again, they would be guaranteed the win. Their search pattern was quite interesting their goal was simply to go through the search area in a spiral pattern while doing reactive obstacle avoidance. It seems their conceptualization of how dense the obstacles would be was more accurate than ours. We had imagined a labyrinth like structure whereas they clearly imagined a mostly open space with convex obstacles. This also meant that with their reactive obstacle avoidance, they could get stuck in the house if their vehicle decided to enter it. Oddly enough and fortunately for us, their quad got stuck in front of some trees while attempting to exit the search area. That meant that it was still possible to beat them if we could perform a moving landing in less time they did.

At this point, its go big or go home. Do or die. Last chance. Go for the gold. You get the point.

Our time comes up and Alex takes the gear to the truck, we wish him luck and send him off. The first runs, something seems off. The truck drives in a loop once and stays at the start point waiting for the quad to come back. Instead of coming back, it seems the quad hesitates between two directions and gets stuck in a wobble loop. As the commentator put it: “It looks like uh, team MRASL is breakdancing in the air”. More attempts and more of the same. No one on the team knows what to tell Alex or what to suggest. We don’t want to go in and disturb is concentration. Tien figures maybe we need to lower the flight altitude. Alex does so and now the quad seems to come back to the truck instead of getting stuck in the air. However as the quad approaches to truck and centers itself on the tags, it oscillates in altitude with an amplitude of about 5 or 10 meters. It seems to go up and down over the truck without landing on it. We have no idea what is going on. Suddenly, DJI tells us our time is up and we have to pack up quick because a storm is coming. What was 45 minutes felt like 5. We had no idea what had just happened.

In the shuttle ride to the Hangar the entire team is under shock. The one thing we knew worked, didn’t work when it had to. We had been convinced we had a good chance at winning the competition and in the end we demonstrated nothing. Alex mustered a “I’m really sorry guys, I don’t know what happened.” We were too tired to even reply. We were beyond disappointed. We were pissed.

Closing ceremony

The closing remarks were one of the most awkward events that weekend. DJI announced that Autero was the winning team and presented them with a check. Autero said a few thank you’s and such but when they got off stage the entire hangar was silent. Jimmy Halliday, obviously super stoked and wanting to break the silence, says to everyone: “Thanks guys, this was fun!” No one replied. Probably not because of jealousy or bad intentions but mostly because everyone was exhausted.

We grabbed some food and decided to pack up and leave ASAP. We left so fast we even missed the group photo they took and the participation certificates they were giving out. As we were walking out of a corridor one of DJI’s media team remarked “man you guys don’t look like you’re celebrating.” Haha dude. We look like a team of 5 guys who missed their chance at 100 000$.

We slept a few hours and met up for drinks (water) with the guys from TU Graz in the hotel lobby. Pretty chill bunch. While we were talking with them someone from DJI came up to me and gave me our participation certificates. Oops.

Want to know what actually happened and what we can take out of this experience? Read on to my failure analysis post!