pinot: delete segment(s) api for an offline table doesn't delete segments from server disc
Use https://docs.pinot.apache.org/users/tutorials/ingest-parquet-files-from-s3-using-spark to create an offline table. I invoked delete all segments api, or delete segment api from swagger with retention 0d or -1d and observed that the segment got deleted from the controller ui and zookeeper but did not get deleted from the server disc. If i used
pushJobSpec:
copyToDeepStoreForMetadataPush: true
in the spark ingestion job spec then the segments got deleted from the s3 segment store but still didn’t get deleted from the server disc.
I tried deleting the table, and even then the segment(s) did not get deleted from the server disc. I do not have a retention configured in the table config for this table.
Even disabling the table, and then invoking delete segment(s) apis didn’t delete segments from the server disc. This is resulting in increased disc usage on daily refresh.
About this issue
- Original URL
- State: closed
- Created 2 years ago
- Comments: 16 (16 by maintainers)
This is a known issue, which is tracked here #9102
Is the problem resolved after removing these 2 extra entries?