Arlen Specter, a moderate ex-senator from Pennsylvania who played key roles in critical Senate battles, but angered colleagues by switching from Republican to Democrat, died Sunday. He was 82. Specter died of complications from non-Hodgkin's lymphoma -- a form of immune system cancer -- at home in Philadelphia, his family said. Former Specter spokesman Christopher Nicholas also confirmed the death. The five-term senator was elected in 1980 as a Republican, but he was one of just three in the party to vote for President Barack Obama's stimulus plan in 2009. After being labeled a pariah by conservatives, he defected to the Democratic Party, only to lose its primary in 2010, ending his long Senate career. "As the Republican Party has moved farther and farther to the right, I have found myself increasingly at odds with the Republican philosophy and more in line with the philosophy of the Democratic Party," Specter told reporters at the time. But Specter also tied the decision to his already faltering re-election bid, angering some Democratic voters. Specter played a key role in several Supreme Court nominations, notably derailing the 1987 nomination of conservative Robert Bork to the dismay of many of his fellow Republicans. But the unpredictable politician angered liberals four years later when he backed the conservative Clarence Thomas in 1991. During president Bill Clinton's 1999 Senate impeachment trial, Specter tried to vote "not proved," citing a precedent in Scottish law, but ended up joining the president's Democratic backers in declaring Clinton "not guilty." Obama endorsed Specter during the senator's 2010 Democratic primary, but it wasn't enough and Pennsylvanians instead chose upstart congressman Joe Sestak, a retired vice admiral with appeal among core Democratic supporters. Sestak lost to conservative Pat Toomey, and the seat stayed in Republican hands. In a statement, Obama hailed Specter as "fiercely independent -- never putting party or ideology ahead of the people he was chosen to serve." The president said Specter brought "that same toughness and determination to his personal struggles," which included a brain tumor and heart bypass surgery, as well as his fight with cancer, which doctors said returned in 2005. "Arlen fought that battle for seven more years with the same resolve he used to fight for stem cell research funding, veterans health and countless other issues that will continue to change lives for years to come," Obama said. Senate Majority Leader Harry Reid, who served with Specter for 28 years in Congress, described his colleague as "a man of moderation; he was always passionate, but always easy to work with," Reid said. "America is better today because of Arlen Specter." Republicans also paid tribute to their onetime colleague. Iowa's Chuck Grassley, elected to the US Senate the same year, praised Specter's "tenacity and willingness to fight hard" regardless of the challenge. "We came from very different places... and we had different views, but we shared a commitment to making the legislative process work in the Senate," Grassley said. Specter, who was Jewish, was born February 12, 1930 in Wichita, Kansas. His father Harry had little formal education; he sold fruit and blankets and ran a scrap yard. But son Arlen excelled in high school and eventually graduated from the Ivy League's University of Pennsylvania, then Yale Law School. Early in his legal career, Specter served on the Warren Commission that investigated president John F. Kennedy's assassination and was behind the controversial "single bullet theory" -- a key underpinning of the panel's finding that Lee Harvey Oswald was the sole gunman. As a senator, he infuriated many Republicans for his support of abortion rights, refusing to cave in to a party that was leaning further and further to the right. Earlier this year, Specter released his memoir "Life Among the Cannibals," which detailed his decision to support Obama's stimulus, how he was shunned by the GOP, and the rise of the far-right Tea Party movement. "'Extremism' was no longer sufficiently extreme to describe what was going on," Specter wrote about the anti-incumbent wave that swept Washington in 2010. "The quest for ideological purity was destroying comity and compromise and bringing government to a standstill." Specter eventually became the longest-serving Pennsylvanian in Senate history. "When you look at his entire tenure in public service, he may have done more to benefit Pennsylvania than any other politician since Ben Franklin," former governor Ed Rendell told The Philadelphia Inquirer.