Australian taxation residency problem?

My problem is that I work for a US company in California that has accepted a US address as my residence. (small apartment) They allow me to work remotely here in Australia. However, they don't want to establish a buisness office in Australia as they have very little buisness here and don't want to pay for the required licenses. Sort of a don't ask don't tell policy. However, since I live here I think I will have to pay Australian taxes and I'm already paying US taxes. If I change my legal residence to Australia with my company they have already said they will just lay me off. So I either pay taxes to both Australia and the US (104% of my income) or I have no income at all. Can I just continue to be a US resident for tax purposes? Who do I contact at the Australian tax office to sort this out?