For emulation on PC, use RPCS3 + official disc dumps; for real PS3, invest in a larger external drive instead of chasing impossible compression ratios.
You will need two free programs:
Instead of chasing “highly compressed extra quality” (which largely doesn’t exist for PS3 without quality loss), consider: ps3 iso highly compressed extra quality