Tips & Tricks: Super Delete S3 Objects

When you need to delete 10s of 1000’s of s3 objects

export BUCKET=ecs-dbt-dev-staging-bucket
export PREFIX="Connecticut/EmplId-Empl"
aws s3api list-objects-v2 --bucket $BUCKET --prefix $PREFIX --output text --query \
'Contents[].[Key]' | grep -v -e "'" | tr '\n' '\0' | xargs -0 -P2 -n500 bash -c \
'aws s3api delete-objects --bucket $BUCKET --delete "Objects=[$(printf "{Key=%q}," "$@")], Quiet=true"' _