Important: Submission policy

Parameter tuning is only allowed on the training data. Evaluating on the test data via this evaluation server must only be done once for the final system. It is not permitted to use it to train systems, for example by trying out different parameter values and choosing the best. Only one version must be evaluated (which performed best on the training data). This is to avoid overfitting on the test data. Results of different parameter settings of an algorithm can therefore only be reported on the training set. To help enforce this policy, we block updates to the test set results of a method for two weeks after a test set submission. You can split up the training data into training and validation sets yourself as you wish.

It is not permitted to register on this webpage with multiple e-mail addresses nor information misrepresenting the identity of the user. We will ban users or domains if required.

 
 

Sample Script and Submission Guidelines

To participate in the evaluation, please submit your algorithm script and the corresponding trained policy models. The process works as follows:

1. Prepare Your Script

Your script should follow the provided sample format. It must take the packed box sequence as input and output the appropriate placement actions. The input and output formats are as follows:

Input:

• A sequence of packed boxes.
• Related configuration information for your algorithm (e.g., model type, hyperparameters).

Output:

• Action: a tuple (rot, lx, ly) representing the rotation and placement coordinates.
• Planning time: the time taken by your algorithm to plan the placement.

You can refer to the sample script files in the PCT-port directory for more guidance. For detailed instructions, please check the readme.md. Specifically, ensure that the format of main.py and pack.py remains consistent with the sample script files, and modify the pack_box function to implement your own algorithm. Be sure to maintain the same input-output format as shown in the sample.

2. Required Files

Along with your script, please provide the following two trained policy models for each data type:

• Ideal Model (corresponding to the ideal conditions).
• Physics Model (considering the physics-based constraints).

3. Submission Process

Send the following materials to our email address (robot-bin-packing-benchmark@gmail.com) for evaluation:

• Your algorithm script in the provided format.
• The two policy models (Ideal, Physics) trained for the respective data types.

4. Test Process

Once submitted, we will use our test data to evaluate your script. The script must:

• Read the test data and the related configuration (e.g., trained model).
• Output the predicted placement actions (rot, lx, ly) and planning time.

Example Input-Output Format:

    Input:
    Box sequence: [box1, box2, box3, ...]

    Output:
    Action: (rotation, lx, ly)
    Planning time: 5.2 seconds
    

Please ensure your script adheres to this format and submit it according to the instructions above.