Arjun pmm, It seems I’ve got the same problem. I also tried rs download and tiler.py on geotiff.

$ cat buildings.tiles | wc -l
435917

$ find masks/20 -type f | wc -l
435917

$ find images/20 -type f | wc -l
469556
; images directory is the result of the tiler.py script

$ comm -12 <(cd masks ; find 20/ -type f | cut -f 1 -d '.' | sort ) <(cd images ; find 20/ -type f | cut -f 1 -d '.' | sort) | wc -l
5510

This means that:

only 5510 images have a corresponding mask;

and

only 5510 mask have a corresponding image;

How many GeoTIFF files did you download from openaerialmap.org (https://oin-hotosm.s3.amazonaws.com/....tif)?