This has been a major annoyance of mine. I have the same model.
It has nothing to do with load on the inverter - i am currently using the inverter (loadshedding at present) with a draw of about 300W and the inverter is QUIETER than when there is no loadshedding, no draw (0W) and batteries are at 100%. In this instance the inverter should just be in trickle charge mode - with barely any cooling necessary - but it blasts it fans on full.
Any feedback that resolves this issue for a Growatt SPF 3000TL HVM-24 would be well received.