Japan quake data delivery is an ongoing challenge

GeoEye post-quake imagery prepared by Google Crisis Response Team

We're part of a great distributed team doing map support for the search and rescue task forces in Japan.  One of the big challenges we've all been working on is how to get updated data to the field. In the early phases of a response you don't know the situation, so you try everything and stick with what works.  In that spirit, here's a story of some of the comm approaches we've worked on.

• Hand off hard drive before departure. This was the first thing we did and so far the one that worked best.  We managed to wire 3 GB of map data to JPL in Pasadena, which was on the way from CA-TF2's staging area to their departure point at March Air Reserve Base. That let our guys at JPL do a physical handoff of a hard drive.  Since then we've heard from the team in the field that they made good use of the data during their flight to pre-plan their operations, so thanks again to everyone who helped make that happen!
• Courier data from Google Tokyo. We figured out on Sunday that there was a solid link between Google's offices in Mountain View, California and Tokyo, sufficient to send ~ 30 GB of data in a few hours.  The plan was to courier the data by hard drive from Tokyo to the operational area at Ofunato, a small town about 100 km north of Sendai. Unfortunately, things stalled there. We've been advised that road conditions are very poor with some bridges out, emergency vehicles and aircraft are all in use, and some roads may be closed to civilian vehicles. As of now we're continuing to stage data to Tokyo and hoping conditions will improve.
• Send data via satellite internet. The team in the field has a satellite dish that uses the BGAN Inmarsat service to give them a direct low-bandwidth internet connection. For now this is our only working data link and we're trying to generate really tightly focused lightweight maps that cover the high priority areas. I don't have the specs on their particular BGAN device but we've been advised that 50 MB is the upper bound file size for practical transfers under good conditions, and unfortunately conditions are poor. That's probably because the satellites are overloaded, just like cell phone networks get overloaded after a disaster.
• Courier data from Misawa Air Base.  Misawa is located at the northern end of the main Japanese island of Honshu and it's where the teams originally arrived and started their drive down to Ofunato. We've established a data link to Misawa and are waiting to hear more about the bandwidth. It may be easier to courier data from Misawa rather than Tokyo, especially because we may be able to piggyback a hard drive in the delivery when they resupply the Air Force security team that's accompanying the search and rescue task force. But so far we don't know when a resupply trip will happen.
• Get help from Cisco. The Cisco Tactical Operations team is investigating a couple of options. One is to deliver data through Cisco's Japan offices and local staff that are providing comms to the Japanese government. Another is to get a higher-bandwidth satellite dish to the field team. We just got in touch with them Monday night.

One really important point about data transfer: the rapid growth of storage media sizes and satellite imagery resolution is an ongoing tech trend and wireless comm bandwidth isn't keeping up. So the trade-off between physical media delivery vs. wireless transfer will probably favor physical delivery more and more as time goes on, and we need to pre-plan how to make that happen for future deployments.

So, hope that is food for thought and I'm sure improving the connectivity will be a fun conversation during debrief.

Calling for mapping help for Japan earthquake

We're helping to provide map support for the FEMA Urban Search and Rescue Task Forces deployed to help with the Japan Sendai Quake.  I'm looking for help or ideas related to challenges listed below.

Please post comments on this blog post where everyone can see them.  Thanks!

FEMA Urban Search and Rescue California Task Force 2 (LA County) staging for the trip to Japan

Challenges:

• Cacheable data sets: We're looking for mapping data sets that can be cached and delivered on a hard drive for offline use.  For reference, the task forces will be splitting into small groups which will have spotty internet connectivity in the field.  All their laptops have at least the basic version of the Google Earth client.  We've already grabbed small areas of OpenStreetMap data, and Google has pulled in lots of data sets, but there may be some great resources we're not aware of.
• Data delivery: We're looking for ways to deliver the data.  We were able to hand off a hard drive Friday evening before they flew out of LA but currently we don't have any pipes.  One possibility is delivering data to Google's Tokyo office and handing off from there, but we haven't worked out the logistics.  Looking for creative solutions.
• Scripting support: The publicly accessible versions of some data sets need to be massaged into something that's usable offline.  If you have behind-the-scenes access to rsync or great map scripting chops to reformat data sets, we might be able to use your help.  An example data set is this GeoEye imagery KML prepared by the Google Crisis Response Team.  How do we cache it?
• File sharing admin support: I set up a 1 TB Apache+WebDAV file sharing server on Amazon Web Services (EBS) to help us collect data.  I haven't configured backups yet.  I am not an admin, there are probably other things I forgot to do. If at this point you are thinking "what a moron", please get in touch :)

Other notes on the situation:

• The latest I have on Task Force 2 is that they will be assessing damage in Ofunato tomorrow morning.
• Ways to get in touch with me for more information:
• Email: trey.smith@nasa.gov
• Twitter: trey9000
• AIM: treyontheoutside
• Google Talk: trey.smith@gmail.com
• Cell: 412-657-3579

GeoCam assisting Japan Sendai quake response

We've been providing map data to the FEMA Urban Search and Rescue Task Forces deploying to Japan to help after the Sendai Quake.  The mission is called Operation Tomodachi (Japanese for "Friendship"). They will likely have spotty connectivity and low bandwidth while deployed, so we need to gather the maps they need up into big chunks of data that can be cached on their local disks.

Tsunami wave of mud hitting a city in northern Japan

Area around Sendai where we cached OpenStreetMap data

Our first data delivery was at around 7:30 PM PST Friday, when we managed to push a bunch of map data to JPL and some friends down there got it onto an external hard drive and handed it off to a member of California Task Force 2 (LA County) before their group departed for Tokyo.  We're still trying to figure out when our next opportunity to send data will be.  For the moment we're collecting data on a server in the  States so we're ready.

This has been a great team effort involving a bunch of NASA folks, OpenStreetMap folks helping us grab their data, Google offering free licensing on their Earth Enterprise product and negotiating data release of Zenrin street and GeoEye satellite data, and even a local Best Buy in Pasadena donating a couple of hard drives.  It is so amazing to work with people who will take that call and instantly drop what they're doing to help out with something like this. I'll have to give it a real write-up when things cool off.