Investigate performance differences between Madagascar and our planner #38

New Issue

patrick · 2017-12-08T17:30:00+01:00

patrick commented

2017-12-08 17:30:00 +01:00

With some domains, we got drastically different results between Madagascar and our planner (generate-and-check configuration).

These domains include:

ipc-2000/blocks-strips-typed (where we are much better)
ipc-2011/elevator-sequential-satisficing (where Madagascar is much better)
ipc-2011/parking-sequential-satisficing (where Madagascar is much better)
ipc-2011/tidybot-sequential-satisficing (where Madagascar is much better)

The following metrics could be of interest in order to investigate this:

grounding vs. solving times with our planner
the number of strongly connected components obtained by Madagascar’s preprocessing
the plan length after which the planners terminated (successfully or not)

With some domains, we got drastically different results between Madagascar and our planner (generate-and-check configuration). These domains include: - `ipc-2000/blocks-strips-typed` (where we are much better) - `ipc-2011/elevator-sequential-satisficing` (where Madagascar is much better) - `ipc-2011/parking-sequential-satisficing` (where Madagascar is much better) - `ipc-2011/tidybot-sequential-satisficing` (where Madagascar is much better) The following metrics could be of interest in order to investigate this: - grounding vs. solving times with our planner - the number of strongly connected components obtained by Madagascar’s preprocessing - the plan length after which the planners terminated (successfully or not)

patrick added this to the Extended Paper for TPLP Journal milestone 2017-12-08 17:30:00 +01:00

patrick self-assigned this 2017-12-08 17:30:00 +01:00

patrick added the

task

label 2017-12-08 17:30:00 +01:00

patrick commented

2017-12-08 18:33:02 +01:00

The number of strongly connected components doesn’t seem to be a good indicator. We were better in ipc-2000/blocks-strips-typed, where the maximum strongly connected component size was 1. However, this was also 1 for ipc-2011/elevator-sequential-satisficing, where we were much worse, and for ipc-2011/parking-sequential-satisficing and ipc-2011/tidybot-sequential-satisficing (where we also were much worse), the size of the largest strongly connected components ranged between 20 and 60.

The number of strongly connected components doesn’t seem to be a good indicator. We were better in `ipc-2000/blocks-strips-typed`, where the maximum strongly connected component size was 1. However, this was also 1 for `ipc-2011/elevator-sequential-satisficing`, where we were much worse, and for `ipc-2011/parking-sequential-satisficing` and `ipc-2011/tidybot-sequential-satisficing` (where we also were much worse), the size of the largest strongly connected components ranged between 20 and 60.

patrick commented

2017-12-13 00:32:40 +01:00

Javier found out that this is likely due to the differences between exists-step and for-all-step planning.

patrick closed this issue

2017-12-13 00:32:40 +01:00

This repo is archived. You cannot comment on issues.