Reading your post brings back "fond" memories of swearing and complaining :-[
The one thing the Telkom guy told me is that I am too far from the exchange. I eventually sort-of confirmed this (ok, I convinced myself that I confirmed this) after doing lots of reading and by comparing the attenuation on my line with some other magical figure, indicating that it is "low". I was advised that I will probably never get 4mbps and that I should rather stick to a slower line speed. This was especially true for reliability of the network.
I constantly ran (and saved) the results from
http://www.speedtest.net/, showing this to the guy when he came to look at the issue.
I get a maximum of 1.6 on a 2mbps line now which seems to be fair. When it was 4mbps it was really unreliable (apparently because of signal loss).
Check this thread on Mybroadband:
http://mybroadband.co.za/vb/showthread.php/186156-Understanding-SNR-and-Attenuation-Rates
Maybe this will help in some way (even though you can see the box from your house ?).
If you get maximum speeds at some point during the day it does seem plausible that the exchange may be overloaded >☹.
I agree, log complaints and keep pestering them!