Awesome

Team Ensemble Radiant Earth Spot the Crop XL-Challenge Solution

Team Ensemble: Brainiac | Dr Fad

Many thanks to the organizers for such an exciting challenge!

Objective

The main objective of this challenge was to use time-series of Sentinel-1 and Sentinel-2 multi-spectral data to classify crops in the Western Cape of South Africa. The challenge was to build a machine learning model to predict crop type classes for the test dataset. The training dataset was generated by the Radiant Earth Foundation team, using the ground reference data collected and provided by the Western Cape Department of Agriculture

Metrics of success

The evaluation metric for this challenge was Cross Entropy with binary outcome for each crop

<img align="left" width="150" height="50" src="https://zindpublic.blob.core.windows.net/public/uploads/image_attachment/image/889/1846525e-af12-45c4-bc2e-4232549842bd.png"><br/><br/>

In which:

• j indicates the field number (j=1 to N)
• N indicates total number of fields in the dataset (87,347 in the train and 35,389 in the test)
• i indicates the crop type (i=1 to 9)
• y_j,i is the binary (0, 1) indicator for crop type i in field j (each field has only one correct crop type)
• p_j,i is the predicted probability (between 0 and 1) for crop type i in field j

Hardware resources

• Google colab pro

Solution Approach

Data Download and Manipulation

• Images were downloaded in batches to avoid out of memory error as colab TPU has a maximum of 35gb RAM
• The images were zipped and stored in google drive.
• Images with a 10 day frequency were used to get the raw image pixels
• Images at the start and end of every month were also processed to raw numpy values
• Pyspark was used to get the mean of the pixel values for each field. Pyspark was utilised becaused the data was quite huge and because of limited compute resources.

Featue Engineering and Preprocessing

• Removed skewness using square root
• Vegetation indices calculations
• Vegetation Indices aggregation - mean
• Vegetation indices differences between different periods
• Quantiles
• Filled missing and infinite numbers with -999999

Model Training

• Catboost classifier trained on vegetation indices data using a 10 stratified cross validation strategy
• LGBM classifier trained on vegetation indices data using a 10 stratified cross validation strategy
• A pytorch classifier trained on raw image pixels.

Final Model

• The final model is an ensemble of boosting trees i.e LGBM and catboost and a pytorch classifier

To reproduce the same score on the leaderboard follow this instructions

1. Upload the Feature_Engineering_&_CATBOOST.ipynb notebook to colab.

   • Enable GPU runtime
   • Run all to get the catboost_models file

2. Upload the Feature_Engineering_&_LGBM.ipynb notebook to colab.

   • Enable TPU runtime
   • Run all to get the lgbm_models file

3. Upload the Pixel_Features-Pytorch.ipynb notebook to colab

   • Enable GPU runtime
   • Run all to get the pytorch_models file

4. Finally upload the Ensemble.ipynb notebook to colab

   • Upload the lgbm_models file
   • Upload the catboost_models file
   • Upload the pytorch_models file
   • Run all to get the final submission file