Joseph incidentally had to resign as a Minister from the V.S. Achuthanandan cabinet in 2006, a few months after the new Left government was sworn in, following a case of his misbehaviour with a woman on board a flight from Chennai to Kochi.
http://ummid.com/news/2011/September/17.09.2011/court_summons_to_kerala_minister.htm
http://ummid.com/news/2011/September/17.09.2011/court_summons_to_kerala_minister.htm
