Repeated over and again in the senator's mild Tennessee drawl, those words guided Americans through the tangle of Watergate characters and charges playing daily on TV to focus squarely on Richard Nixon and his role in the cover-up.
Baker's famous question has been dusted off for potential White House scandals big and small ever since.
Baker, later became Senate majority leader, chief of staff to President Ronald Reagan and one of the GOP's elder statesmen died Thursday at his Tennessee home of complications from a stroke suffered days earlier, according to an email distributed at the law firm where Baker was senior counsel. He was 88.
Baker emerged as an unlikely star of the Watergate hearings in the summer of 1973.
When chosen as vice chairman - and therefore leading Republican - of the Senate special committee, he was a Nixon ally who thought the allegations couldn't possibly be true. Democrats feared he would serve as the White House's "mole" in the investigation of the break-in at Democratic headquarters and other crimes perpetrated in service to Nixon's re-election.
"I believed that it was a political ploy of the Democrats, that it would come to nothing," Baker told The Associated Press in 1992. "But a few weeks into that, it began to dawn on me that there was more to it than I thought, and more to it than I liked."
He said Watergate became "the greatest disillusionment" of his political career.
Baker's intense but restrained style of interrogating former White House aides played well on camera. A youthful-looking, side-burned 47-year-old, his brainy charm inspired a raft of love notes sent to his Senate office; a women's magazine proclaimed him "studly." He was mentioned frequently as presidential material.
By the time Nixon resigned in 1974, Baker was a household name with a reputation for fairness and smarts that stuck throughout a long political career.
Howard Henry Baker Jr. had a fine political pedigree - his father was a congressman from Huntsville, Tenn., and his father-in-law a prominent senator from Illinois. Over the years, his name would be knocked about for big Washington jobs including vice presidential candidate, Supreme Court justice and CIA director. But his focus remained on the Senate and, at times, the White House.
In 18 years as a moderate Republican senator, he was known for plain speaking and plain dealing. He had a talent for brokering compromise, leading some to dub him "the Great Conciliator."
"Senator Baker truly earned his nickname: the Great Conciliator. I know he will be remembered with fondness by members of both political parties," Sen. Mitch McConnell of Kentucky on the Senate floor Thursday, announcing Baker's death to the chamber.
Tennessee Sen. Lamar Alexander, who once worked as an assistant to Baker, called him "Tennessee's favorite son" and "an indispensable friend."
"He built our state's two-party political system and inspired three generations to try to build a better state and country," Alexander said in a news release Thursday.
Baker was minority leader when the Reagan landslide swept Republicans into control of the chamber in 1980 Reagan, and he became the first Republican majority leader in decades.
Putting aside his own reservations about Reagan's economic proposals, Baker played a key role in passage of legislation synonymous with the "Reagan Revolution" - major tax and spending cuts combined with a military buildup.
Baker considered his years as Senate majority leader, 1981 to 1985, the high point of his career. He called it "the second-best job in town, only second to the presidency."
He made a fleeting bid for that best job in 1980, and left the Senate with an eye to another presidential run in 1988. Instead, he ended up in the White House as Reagan's chief of staff.
Reagan needed him to put things in order after ousting chief of staff Donald Regan amid scandal over the administration's secret moves to trade arms for hostages in Iran and divert the profits to Nicaraguan rebels - another of history's what-did-the-president-know moments.
Baker recalled marshaling all his reasons for refusing the offer, but he couldn't turn down Reagan. "I guess I am a pushover for presidents," he said.
The Reagan White House weathered Iran-Contra. But Baker lost his last chance at the presidency.
"I have seen it up close and personal and I am convinced that I could do that job," he said. "But that boat never came to dock."
During much of the 1980s and '90s, Baker grappled with the illness of his wife, Joy, daughter of Everett Dirksen, a former GOP Senate leader. She died in 1993 after an 11-year battle with cancer. The couple had two children.
In 1996, Baker married Kansas Sen. Nancy Landon Kassebaum. It was the first time two people who had served in the Senate married.
President George H.W. Bush sent Baker to Moscow in 1991 to meet with Soviet leader Mikhail Gorbachev before a summit; George W. Bush named him ambassador to Japan in 2001.
An accomplished amateur photographer, Baker carried a camera wherever he went. But he didn't take any photos during the Watergate hearings.
"I felt that it was beneath the dignity of the event," he said years later. "It turned out the event had no dignity and I should have taken pictures."