New Delhi/Mumbai, Jun 20 (Reuters): Indian central bank governor Raghuram Rajan's abrupt decision to quit came as he increasingly felt he lacked support from his political bosses Finance Minister Arun Jaitley and Prime Minister Narendra Modi, according to friends and colleagues.

A newspaper report a week ago that a selection panel would consider a field of candidates rather than directly offer the former IMF chief economist an extension to his three-year term, effectively forcing him to reapply for his own job, may have been the final straw, according to these people and a finance ministry source.

"He felt it would belittle the position of the RBI governor if he had to appear before the committee," said one senior commercial banker who knows Rajan personally but had not spoken to him since his decision.

"It would reveal a lack of government support. Rather than have two more years of constant quibbling, he decided to go."

The darling of international investors was also upset that Jaitley had not backed him more strongly after criticism from Hindu nationalists of both his policies and his perceived lack of "Indian-ness", the sources said.

Rajan joked when he took over the top job at the Reserve Bank of India in September 2013 that he wasn't expecting to win any votes or Facebook 'likes' in the position.

But the hostility he had faced of late from elements of Modi's ruling party was evidently greater than he had counted on, the sources said.

When Rajan decided to leave, he did so without warning and on his own terms: in a sign of growing tensions he did not inform top members of the government before releasing an open letter to staff on Saturday, a move that took investors and the government by surprise.

No contact

In his letter, Rajan said he had been open to seeing through changes he championed such as the creation of a monetary policy committee to set interest rates and a clean-up of the banking sector.

But he added: "On due reflection, and after consultation with the government, I want to share with you that I will be returning to academia when my term as governor ends on Sept. 4, 2016."

Rajan was not available for comment but will face questions on the motives for his abrupt withdrawal at a speaking engagement on Monday afternoon in Mumbai. Less than two weeks ago, he laughed off speculation about his future after an RBI policy meeting, saying: "You will know when there is news."

Five sources familiar with the matter said Rajan had not contacted Modi's office directly. Nor was a meeting or discussion between the two planned, said one senior official.

Modi has not commented, tweeting instead on Sunday about the forthcoming World Yoga Day.

Jaitley had been in his office on Saturday and gave no indication of concern about Rajan's future, leaving for home before the news broke, an aide said.

The finance minister tweeted after a delay of more than two hours that he respected Rajan's decision.

It was unclear whether Rajan had briefed senior staff at the Reserve Bank of India since his bombshell decision.

"I am surprised," said one person who has worked with Rajan and who spoke on condition of anonymity. "A week back he was sounding very much interested in serving a second term."

Saffron posse

Rajan had for weeks faced intense pressure from Subramanian Swamy, a politician in Modi's Bharatiya Janata Party (BJP) who, say sources familiar with the matter, was acting with the backing of the Hindu-nationalist umbrella group to which the BJP is affiliated, the Rashtriya Swayamsevak Sangh (RSS).

Swamy, a 76-year-old former Harvard economist with a record of aggressive anti-corruption litigation, wrote an open letter to Modi a month ago accusing Rajan of being "mentally not fully Indian" and calling on the prime minister to "terminate" him with immediate effect.

The broadside from Swamy came soon after he was handed a seat in India's upper house of parliament by the ruling party, creating the impression that he was acting on good authority.

Challenged to respond to Swamy, Jaitley merely said that Rajan's candidacy should not be discussed publicly.

"As a colleague I could see he looked hurt," said one senior policymaker who works with Rajan. "You can question my efficiency, but if you question my loyalty to my job, then one would expect your employer to stand up and defend - and not stay silent."

Yet it was Rajan's own forays into politically sensitive territory that led the Hindu right to target him - in particular over a speech last October to students in Delhi in which he said that social tolerance was vital for a country's development.

"No-go zone"

The issue of "intolerance" has been a catalyst of controversy under Modi, driving to the heart of whether the overt message of development for all that he campaigned on to win the 2014 general election can hold, or whether a more divisive Hindu-first agenda will prevail under his party's rule.

Questions about where Modi stands on the issue date back to early in his term as chief minister of Gujarat, where more than 1,000 people, most of them minority Muslims, died in communal rioting in 2002. He was dogged for years by allegations that he either turned a blind eye to, or even encouraged, the attacks until a court-ordered probe found there was not enough evidence to prosecute him. Throughout, Modi has denied wrongdoing.

"It was Modi who protected him for the longest time but the defence wall collapsed when Rajan entered the no-go zone of politics," one senior RSS official told Reuters, speaking on condition of anonymity.

After he was appointed by the last Congress government, the 53-year-old Rajan took extended leave from his post as a finance professor at the University of Chicago's Booth School of Business. This would have lapsed had he not returned in time for the fall semester.

Friends also say his family had wanted him to return - his wife has remained teaching in Chicago.

One senior official said Rajan had signalled his willingness to work until next March - the deadline he had set for completing a clean-up of bad debts in the banking sector. This had not been seen as workable, however, with a "natural" departure at the end of his term rather than an "unnatural" exit later viewed as a less bad option.