Connected Vehicles and Location Based Services Mobile Apps send location updates to the cloud while moving to generate a series of waypoints. Waypoint data has become an important source of information for many industries but especially for transportation analysis.

Data from millions of vehicles or mobile phones can contain hundreds of billions of waypoints. Before you can look at the vehicle miles traveled in a county, perform an origin/destination analysis for a city or review the travel times on a road you need to match every waypoint to the roads and areas. Assigning roads and areas to billions of waypoints can take months of server time and this is usually done running many servers in parallel. Moonshadow is now introducing tools to speed up road and area assignments by 20-50x; MMZIP. What took months of server time before can now be done in a few days.

Figure 1. Otonomo connected vehicle data waypoints
Figure 2. Otonomo connected vehicle data waypoints matched to road lines
Figure 3. Otonomo connected vehicle data waypoints matched to neighborhood areas

MMZIP reduces the size of waypoint data by over 90% lossless. As a result data transfers can be done in one tenth of the time and cost much less. MMZIP is an intelligent compression technology in that it can enrich and filter the data during decompression. Those enrichments now include road assignments and area assignments. Road matching is done at speeds exceeding two million waypoints per second whereas area matching performance exceeds ten million waypoints to area matches per second. The area assignment speeds depends on the size of the area; the larger the area the faster the assignment. The following table shows area assignment speeds for a dataset with 100 million waypoints

MMZIP Area Assignment Speeds
in Million of Waypoints per Second		 AVX No AVX
Census Blocks                             7.1                             6.0
Census Block Groups                           11.2                             9.7
Census Tracts                           12.5                           10.8
Counties                           16.7                           16.6
States                           19.9                           19.7
All Five Area Assignments                             2.9                             2.6

These test results were measured on a computer with an Intel Core i7-4790K CPU running at @ 4.00GHz with 32GB of RAM. MMZIP uses a streaming system so it can enrich datasets that are much larger than the available amount of RAM. MMZIP is available as a command-line tool as well as a C-library that customers can fully integrate into their own waypoint data processing pipeline on their own servers.

This test included an area with 1.9 million Census Blocks and we see that MMZIP area assignment is much slower on area meshes with a large number of small areas such as Census Blocks than with larger areas such as Census Block Groups. MMZIP runs on Intel CPUs and will use AVX if it is available and the table shows that AVX provides a 15% or better speed on the smaller areas. For county and state assignments AVX does not provide a significant benefit.

Please email wander@moonshadow.com for more information.