Taxi Trip Time Winners' Interview: 3rd place, BlueTaxi

Taxi Trip Time Winners' Interview: 3rd place, BlueTaxi

This spring, Kaggle hosted two competitions with the ECML PKDD conference in Porto, Portugal. The competitions shared a dataset but focused on different problems. Taxi Trajectory asked participants to predict where a taxi would drop off a customer given partial information on their journey, while Taxi Trip Time's goal was to predict the amount of time a journey would take given the same dataset.

418 players on 345 teams competed to predict the time a taxi journey would take.

Team BlueTaxi finished 3rd in Taxi Trip Time and 7th in Taxi Trajectory. This blog outlines how their team of data scientists from five different countries came together, and their winning approach to the Taxi Trip Time competition.

The Basics

The BlueTaxi Team

The BlueTaxi team is very multicultural, we are Ernesto (El Salvador),Lam (Vietnam), Alessandra (Italy), Bei (China), and Yiannis (Greece). We had great fun participating and winning the third place in this ECML/PKDD Discovery Challenge which was organized as the Kaggle, Taxi Trip Time, competition. In this post we would like to share with you how we did it.

What made you decide to enter this competition?

Ernesto: The Discovery Challenges, organized annually by ECML/PKDD, are always very interesting and this year was not the exception, especially this one organized on top of Kaggle. After a small chat with Lam, we decided to go for it and enter as the BlueTaxi team for both tracks of the challenge.

I took the lead for the trip time prediction and Lam led the destination prediction.

We invited Alessandra, Bei, and Yiannis to consolidate the final team of five.

Lam: I decided to enter because the competition was hosted by Kaggle and the ECML/PKDD 2015 conference will be a great a opportunity for researchers to benefit from exchanging ideas and their experience during the workshop session being held in conjunction with the main conference in Porto next September.

Alessandra: Lam and Ernesto presented the challenge to me and I thought it was very interesting so I decided to join.

Bei: For me that was also the case.

Yiannis: The problem of predicting the destination and trip time for a taxi route seemed very challenging and that's why I decided to participate and join the BlueTaxi team.

What was your background prior to entering this challenge?

Lam: I did my PhD in pattern mining for data streams at Technische Universiteit Eindhoven (TU/e) and joined IBM Research Lab in Ireland about a year and a half ago. My research interests include mining big and fast data on big data platforms with applications in telcos, transportation under the smarter city project.

Alessandra: My background is in transportation analytics. Specifically I work on estimation and prediction of traffic and urban traffic control systems. I hold a PhD degree in Information Technology from Politecnico di Milano and currently I am a Research Scientist at IBM Research - Ireland (also known as the IBM Smarter Cities Technology Centre in Dublin, Ireland.)

Bei: I am a statistician with primary interests in time series analysis, forecasting, resampling/subsampling methods for dependent data and financial econometrics. My recent work focuses on statistical methods in urban applications. I received my PhD in Statistics from the University of Waterloo, Canada. I joined IBM Research, Ireland, in late 2012.

Yiannis: I am a Research Software Engineer at Smarter Cities Technologies Center, IBM Research - Ireland. I hold a Masters Degree from Athens University of Economics and Business in Computer Science. Lately, I have been working with spatio-temporal data on various projects focusing on data curation, efficient storing and analysis. Moreover, I have experience with visualisation of similar type of data, helping data scientists to gain insights.

Ernesto: I hold a PhD in Computer Science from the L3S Research Center in the University of Hannover, Germany. My background is in supervised machine learning applied to Web Science, Social Media Analytics, and Recommender Systems. I joined IBM Research, Ireland, early 2014.

Do you have any prior experience or domain knowledge that helped you succeed in this competition?

Lam: In the IBM Research lab I have been working on several projects with similar data, e.g. with GPS traces from buses used for prediction of bus arrival time at bus stop, e.g., see our related paper regarding this topic (Flexible Sliding Window for Kernel Regression Based Bus Arrival Time Prediction) in the industry track at ECML/PKDD 2015.

Ernesto: I did not have any particular domain knowledge in transportation systems, but my experience in machine learning, data science and analytics were of course valuable for the competition.

Alessandra: Yes, my knowledge in the transportation field helped me in the challenge.

Yiannis: My experience with spatio-temporal data helped me in the competition.

Bei: I had some experience analyzing transportation data, which helped me in this challenge.

How did you get started competing on Kaggle?

Ernesto: I joined Kaggle a couple of years ago during my PhD. I did enter in some competitions before. The datasets available in Kaggle are usually very interesting and some of them were very useful for my research. But to be honest, I never got good traction in a competition until this time.

Lam: I also joined Kaggle 4 years ago when I was a PhD student. But I haven't tried to compete since then.

Bei: I joined Kaggle in 2012, and this was my second competition since then.

Alesandra and Yiannis: For us this was the first time that we had entered a Kaggle competition 

Let's get technical

What preprocessing and supervised learning methods did you use?

First we created our local training set by selecting the cut-off times the same as the five snapshots on the test set (the same week-date as well). We also observe that 14th of August is the day before a big holiday in Portugal and 21th of December is the last Sunday before Christmas, both very particular days.

We created a bunch of features as follows:

1. Features from 10-NN. For every test trip we find 10 nearest neighbours w.r.t the Euclidean distance and consider the durations of those trips as predictors.

2. Features from Kernel Regression. Similar to 10-NN, kernel regression was used to predict the duration. Remember that kernel regression is a smooth version of kNN and these features yield very good results.

3. Some features from the partial trips: travelled distance, number of GPS updates, last GPS coordinates, average speed at different part of the trips and accelerations at different part of the trips.

When matching a test trip with the training trips, we only consider to match the last 100, 200, 300, 400, 500, 1000 meters and the full trips as well. This is because the later part of the trip is more important in some cases. Our model show that the last 500 meters of the trip is very important.

Two trips with different starting points but with the same destination (Porto Airport). The later part of the trajectories are very close to each other. Therefore via trip matching we can guess destination of the other trip if we can guess the destination of trips with similar route.

We also considered contextual matching (match only trips with same taxi id, the same week date, the same call id, etc.) because we observed different distributions of destination for these contexts. The kernel regression on taxi id context produced the best results.

When modelling, we did not predict the duration of the whole trip but instead predict the additional delta travel time with respect to the cut-off timestamp. Since the evaluation metric was RMSLE, we log-transformed the time target labels, i.e., the log of the additional delta time.

Outlier handling: we found that trips with missing values (identified at speed limits 160, 140, 100 Km/h) are more difficult to predict, we try to recover this information on the test set by looking at the gap between the cut-off timestamp and the start timestamp. Unfortunately, this information is not reliable so we decided to remove outliers based on the number of GPS updates based on an absolute deviation from the median of 3.5.

The test dataset for this competition was very small (320 instances), which makes it very prone to overfitting. Our final solution was a robust Ensemble of several models that included: Random Forests, Gradient Boosted Trees, and Extremely Randomized Trees.

BlueTaxi the overall winner approach ? We expected some variation in the top ranking positions given the small test set, but were quite surprised on all the shuffling we observed from the public to the final private leaderboard. Note that if we average the performance in the 100% of the test set (public and private scores), the BlueTaxi approach is actually the most robust and with the best overall RSMLE score ?

What was your most important insight into the data?

There are many interesting patterns found during data exploration, e.g., some trips with the same call id are very regular, trips to the airport are very easy to predict as well.

We used some open source journey planners to predict travel time but the results, using for example the destination prediction results from the other track of the challenge, but we found that the results were not very good for the taxi trips. This tells us that journey planners should include the real-time traffic information to improve its estimation of travel time.

Heatmap of destinations of 155 trips with call id 59708. There are only 8 major destinations. Using call id information one can narrow down the possible destinations of the given taxi.

Which tools did you use?

R, Python (Numpy, Scipy, Sklearn and Pandas), Matlab, and Javacript and Leaftlet for visualisation.

How did you spend your time on this competition?

A rough estimate would be 80% for feature engineering, 10% in data exploration, and 10% in modelling.

What was the run time for both training and prediction of your winning solution?

Most features were based on kNN search, which was very time consuming in general, but we implemented a novel indexing technique to solve the efficiency issues, the running time was reduced significantly with the index.

For the modelling part, the most time consuming step is always the model parameter selection, which we conducted using cross validation. The final training is relatively fast. We included in our ensemble Extremely Randomized Trees, which given their nature are much faster to train with respect to Random Forest or Gradient Boosted Trees, and their individual results were quite competitive in our validation set.

Words of wisdom

What have you taken away from this competition?

Our data-driven and machine learning solution in the end led to significant better results than other models based solely on transportation domain knowledge.

We learnt that data exploration, feature extraction and engineering are very important. Unfortunately, this task still requires a lot of human effort. Modelling is much easier now with many open-source off-the-shelf machine learning tools.

This fact encourages us to continue working on research directions that try to find a generic solution to automate or at least semi-automate this process, i.e. make ML and AI closer to each other.

Do you have any advice for those just getting started in data science?

Be always curious and open your mind to what the data has to tell you.

Make data science your passion!

In Kaggle competitions, what it really matters is your passion to learn and discover new things and not what you already know.

Teamwork

How did competing on a team help you succeed?

We think that competing as a team was the fundamental key for success. Remember that we were participating in both tracks of the ECML/PKDD challenge. Basically two Kaggle competitions at the same time and this was our side project and hobby.

In the end, our BlueTaxi finished 3rd in the taxi trip time prediction and 7th in the destination prediction track. We would not be able to achieve such high rankings if we were not competing as a team.

Bio

Ernesto: Ernesto is passionate about Web Science, Big Data analytics and the power of Machine Learning to support decision making processes. His research focus is to contribute novel and intelligent filtering approaches that leverage social interactions, multidimensional relationships, and other ubiquitous data in the Social Web in order to relieve people's information overload. Ernesto's research has advanced the state-of-the-art in personalized ranking and recommender systems and has demonstrated to be valuable for industrial practitioners as well. Ernesto holds a PhD from the L3S Research Center in the University of Hannover, Germany, and currently is a Research Scientist in Machine Learning and Data Science at IBM Research - Ireland.

Lam: Lam did his PhD in pattern mining in data stream at Technische Universiteit Eindhoven in December 2012. He joined IBM research lab in Dublin Ireland right after that and has been working on different research projects, proposing novel machine learning, statistical modelling approaches to solve real-world problems to improve city life.

Alessandra: Alessandra Pascale received the B.Sc. degree (with honors) from Politecnico di Bari in 2006 and the M.Sc. degree (mark 110/110) from Politecnico di Milano in 2009, both in Telecommunication Engineering, and the Ph.D. degree in Information Technology from Politecnico di Milano in 2013. She was also enrolled in the ASP (Alta Scuola Politecnica) program, a school of excellence between Politecnico di Milano and Politecnico di Torino. She was visiting researcher at University of California, Berkeley, in the TOPL group, from February to July 2012. In 2013, she worked for 6 months in the framework of the European research project DIWINE, "Dense cooperative wireless cloud network".

At the present time she works as Research Scientist at the IBM Smarter Cities Technology Centre in Dublin, Ireland. Her research interests are in the field of signal processing, in particular statistical estimation and prediction of vehicular traffic, distributed/cooperative estimation approaches and urban traffic control systems.

Bei: Bei Chen received her BMath in Statistics and Actuarial Science (with Dean's Honour's list and President's Award), MMath and PhD in Statistics (with Outstanding Achievement in Graduate Studies Award) from the University of Waterloo, Canada. She is the recipient of the 2011 Pierre Robillard Award of the Statistical Society of Canada, which recognizes the best thesis in probability and statistics defended at a Canadian university.

Prior to joining IBM, Bei Chen was an tenure-track Assistant Professor (September 2011 - October 2012) in the Department of Mathematics and Statistics at McMaster University, Canada. Her research was funded by various grants, including NSERC, ECR and MITACS.

Yiannis: Yiannis Gkoufas holds a Masters Degree from Athens University of Economics and Business in Computer Science. He is working as a Research Software Engineer on the High-Performance Systems Group in Dublin,Ireland since 2012. His main focus is building domain-specific applications and architectures on top of distributed computing frameworks. Moreover, he is responsible for developing the middleware supporting web-based applications. He has been involved in a vast variety of funded projects evolving around Energy and Demand Forecast Analytics, Transportation Monitoring and Planning and Event Detection from Social Media.

Disclaimer

Disclaimer from the authors: This post is our own and doesn't necessarily represent IBM's positions, strategies or opinions.