The Gilded Age Billionaires, Part II

Featured image
A nighttime scene in Central Park scene — across the street (Fifth Avenue) from the Whitney house on 79th Street and Fifth Avenue. Photo: JH.

Wednesday, July 26, 2023. A bright sunny warm day, not too hot with the weatherman forecasting a tropical-like (don’t ask) storm and suddenly. It happened somewhere around 3 pm.

It wasn’t as wild as we were led to believe it would be, but weather is often drama and the drama queens acquire wisdom or so it would seem. The storm we got was like all of them this year. Mainly heavy, massive rainfall … for about a half hour. And then Stop; the Sun comes out and dries up the pavement and the road. Everything but some of the curbside puddles; just like life (if you’re lucky).


Making the most of the rain.

Today we are running the second part of a short history on the mansions that were built a century and more ago by the richest families in New York from the last quarter of the 19th century right up to the 1920s. Then the younger generation unlike all that came before, came of age. It was the beginning of a social transformation (which in some ways is still going on a century later). Just like the automobile replaced the horse.

We’re reminded of that era in illustrations and photographs that are fascinating architecturally. It was the beginning of women’s liberation. And men’s too although that’s not as interesting. Today’s chapter is basically about a family who came to the fore at the turn of the 19th to the 20th century. It had its proper background (for social acceptance purposes: Wasps) as well as social behavior (at least publicly). And the real estate was always Real estate.

This is a family story, not uncommon in the roles. But always interesting in the Drama. What interested me was the “forebear” who lived into the 20th century and leaving behind a 20th century sort of life. Or drama. Rich; mega. An enticing drama with a movie star handsome man and a sensational and attractive woman. Or rather, women.


Recap: Part I was about the block on Fifth Avenue between 78th and 79th Street covered the houses on the corners, that of Isaac Fletcher on 79th and James B. Duke on 78th. We resume with the houses in the middle:

Right next door to the Fletchers (and eventually the Dukes who came later) is the house that has the most interesting family history — the Payne and Helen Whitney residence, now the Cultural Services building of the Embassy of France, at 972 Fifth. The five-story mansion was commissioned in 1902, a wedding gift from Mr. Whitney’s maternal uncle Col. Oliver Payne, designed by Stanford White.

The Payne Whitney house at 972 Fifth, completed in 1906.

Work on the Italian Renaissance palazzo-style house was begun in 1903 and completed after White’s murder in June 1906. Separately, right next door was a narrower house, also designed by White and originally contracted by Henry Cook who died before it was completed. It was eventually owned by Col. Payne although he rarely used it. It remains one of the two or three private mansions still standing on Fifth Avenue.

The Whitneys were then one of the richest families in the country thanks to Payne Whitney’s father, William C. Whitney, and most especially thanks to his uncle Col. Payne. They possessed fortunes timely and in full bloom. These houses in the middle of the block ironically turned out to be a milestone in a pronounced family breach that had developed between Mr. Whitney and Mr. Payne.

The final break between the lifelong friends and brothers-in-law had occurred less than a decade before after the death of Whitney’s wife (who was also Payne’s sister). It was over a woman, a young woman (32) amongst these older (50 year old) men. The result of this breach involved one of the largest 19th century American fortunes, and what has remained in the past century, one of the few to last into the 21st century. And the breach between the two men, friends from youth, was never repaired.

William C. Whitney in his 30s.

Born in 1841, William C. Whitney was a bright, handsome and ambitious young man from Conway, Massachusetts. He was descended from William Bradford who arrived on the Mayflower in 1620. Oliver Payne’s father was also descended from Bradford. The Conway Whitneys were not poor, but not rich. William went to Williston and then Yale. He was a very popular student, charming and surefooted and bonus goodlooks.

One of his professors remembered him as “magnanimous, unselfish and generous. He was not at all easily offended …. He made large allowances, or he paid no attention to those aspects of occurrences which might have been expected to touch his personal feeling.” The professor was being generous also, adding: “We all agreed, when we graduated that his success would depend on whether anything would stimulate him to a full development of his powers.” The professor overlooked the power of one crucial characteristic: the politician. Anything or anyone.

William C. Whitney had met Oliver Payne at Yale. Also an agreeable man, Payne was two years ahead of Whitney. He was the son of a rich Cleveland businessman. Payne left Yale to join the Union Army in the Civil War where he was later decorated with his rank.

The two men continued to see each other. After Yale, Whitney moved on to Harvard Law and Payne went on to serve his family’s business interests which eventually merged into another (high) schoolmate’s business: John D. Rockefeller and the Standard Oil Company. Col. Payne became the company’s treasurer. His interest in Mr. Rockefeller’s company was comparable to that of the earliest investors in Microsoft (or Berkshire Hathaway) today in terms of future profits.

Flora Payne Whitney was vivacious and social but without the beauty to match her husband’s good looks.

In 1868, Payne introduced Whitney to his younger sister, Flora. Flora was very bright, like her brother, expressive and vivacious. Flora was not a beauty. Whitney was taken by her, and she by him. They married the following year.

The marriage was a seminal move for young Whitney. He was already very much a man on the rise, but the Payne fortune changed everything overnight. Oliver Payne gave them a house in Murray Hill as a wedding gift. Two years later he bought them the Stevens mansion on the southwest corner of 57th and Fifth (where Bulgari is today). Payne, who never married, was remembered for his “sweetness” and generosity, especially with his family members and their off-spring. There was always the sotte voce suggestion that the Colonel was unrequitedly in love with his brother-in-law. The fact that he never married added an ingredient. Whatever it was, he was devoted to his sister and her family (and his other nephews and nieces as well).

After graduating from Harvard Law, Whitney had moved to New York City where he joined a law firm and found that element his Yale professor said would be required to stimulate. It was New York City. He was a smart lawyer, an attractive go-getter, and soon involved in Democratic Party.

Whitney had everything at his fingertips and he used all of his assets optimally. His political star rose right along with his business ventures. He became associated with Samuel Tilden, who called him the ablest political associate he ever had, and he eventually entered national politics through Grover Cleveland’s Administration where he was Secretary of the Navy. His wife proved to be an able asset in many ways too. When Flora and William Whitney were in residence in Washington, they entertained literally thousands of people at their house. It was reported that over the four-year period they were in residence they entertained more than 60,000 (yes!) people.


The house at 972 contained 40 rooms including Stanford White’s mirrored first-floor reception room later to become known as the “Venetian Room.” The mansion’s lavish interiors were used in the film Rebecca.
The “Venetian Room” today. Photo: Beowulf Sheehan

When Whitney returned from his first Washington foray he got involved in the Metropolitan Railway Company, becoming co-owner of the city’s first rapid transit line, as well as being profitably associated with various public utilities companies. His smoothness, his lack of (in his professor’s words) “attention to those aspects of occurrences which might have been expected to touch his personal feeling” drew attention that was not always favorable.

In short, his detractors — and there were many — regarded him as a sharpie. He’d made a fortune fast and fat, and also acquired the ironic title of “Grandee of Graft.” He also acquired a great deal of real estate including 90,000 acres in the Adirondacks, of which a large portion remains in the hands of his grandson Cornelius Vanderbilt Whitney’s last wife, Marylou Whitney Hendrickson. He also owned houses and thousands of acres in the Berkshires, on Long Island and South Carolina, as well as a cottage in Newport and a famous racing stable. And everyone knew him. And he knew everybody.


The Fifth Floor Studio, originally a study for Mrs. Whitney. McKim, Mead & White, courtesy of the Museum of the City of New York. After the house was sold to the French, this room was removed and stored away and forgotten. When Mrs. John Hay Whitney died the room panels were found packed away in a storage on the Whitney estate.

Flora and William Whitney had five children, losing one, Uncle Oliver’s namesake, at age five, leaving two boys — Harry Payne Whitney who later married Gertrude Vanderbilt, and William Payne Whitney; and two daughters, Pauline Payne Whitney, who grew up to marry an English aristocrat and move there; and Dorothy Payne Whitney, who grew up to marry a man named Willard Straight, with whom she built a house designed by Delano & Aldrich and still standing (now owned by billionaire Bruce Kovner) on 94th and Fifth.

Edith Randolph, later Edith Randolph Whitney.

The marriage of the golden couple, Flora and William, was affected adversely by all this success and activity. Or maybe it would have been, no matter what. Because William Whitney, like a lot of men in his politically and financially powerful position, played around. In about 1890, he became enamored with a very beautiful 36-year-old widow named Edith Randolph. The willowy dark haired Mrs. Randolph was the daughter of a Washington surgeon named John Frederick May. Dr. May had removed a tumor from the neck of John Wilkes Booth, and in 1865 identified the actor’s body by the scar. Mrs. Randolph’s mother, the former Sarah Mills, was related to New York socially prominent Oelrichs, Jays, Winthrops and Kanes. All first rate. Mrs. Randolph’s life, however, turned out to be the stuff of Edith Wharton novels.

The Mays moved to New York in the 1860. Edith’s future was formed. She was in the original “400” list of Mrs. Astor. When she was 20, in 1878, she married a British military captain. A little more than ten years later, with two small children, living in adequate but not luxurious circumstances on an estate in Queens, Mr. Randolph died. He was not a man of means. The pickings could be slim at that age for a woman with children.

Helen Hay and her bridesmaids on the day of her wedding to Payne Whitney.

Her social connections, however, kept her in the swim. She was a very likeble and goodlooking woman liked by both the men and women. She was invited and she participated. JP Morgan became infatuated with her. He was in his early 50s and long married. She was a perfect mistress for a Victorian gentleman. Having her “Mrs.” she was legitimate and could be invited anywhere. As a result she was often present in the Morgan families excursions abroad or yachting up the East Coast to Maine.

Whitney had met her too, and was taken. She was well liked by both husband and wife until … it became obvious to wife that husband was paying too much attention. Flora Whitney had it out with her husband. The lady must go. The lady went.

That was when Mrs. Randolph picked up with Mr. Morgan. Their misalliance lasted for a few years until Mrs. Morgan demanded a stop. It stopped.

Flora Payne Whitney died in 1894 at the age of 52. It wasn’t long after that her widower, William C. was seen in the company of Mrs. Randolph, recently banished by Mrs. Morgan. This was a problem.

Oliver Payne had been well aware of his brother-in-law’s affair with Edith Randolph a few years before. He knew how his sister Flora felt about this bit of business and he resented Whitney’s arrogance and, from his point of view, disloyalty. So when, after Flora Whitney’s death, and Will took up with Mrs. Randolph again, Oliver Payne was outraged.

He insisted that Whitney give the woman up. Whitney refused. Payne turned to his Whitney nieces and nephews. Come with me and you will inherit my fortune, he told them. Go with your father and you will get nothing. Of the four, two went with uncle and two went with father. With uncle was William Payne and his sister Pauline. With father went Harry Payne and his sister Dorothy Payne. When son William went with uncle, he dropped the “William” (his father’s name) and took Payne as his first name. From then on, in the family, he was known as plain Payne.

Payne Whitney.

William Whitney married Edith Randolph in 1896 which sealed the breach between the brothers-in-law. The children remained friendly with both uncle and father. It would almost seem as if the arrangement was agreeable to William C. also who divided up his fortune mainly between the two who had taken his side.

The marriage of William Whitney and Edith Randolph was star-crossed, as if Payne’s damning took hold on their fate. In February 1898, only two years into the marriage, the new Mrs. Whitney had a terrible accident on her husband’s Aiken, South Carolina estate. While riding a tall horse, she came up on a covered bridge which required her to duck while passing under. She didn’t duck quite enough and smashed her head into the overhang, falling to the ground, bleeding and unconscious.

She came out of her coma three days later, having broken cervical vertebra, with both arms paralyzed. Two months later they removed the injured woman to New York where she was confined to bed. She was in constant pain but she kept up correspondence and saw friends and moved to the country in the summertime. By the following spring, she was still confined but sitting up to watch her husband’s horses race at the Meadow Brook Steeplechase. Not long after, she fell into a coma. She died on May 6, 1899, age 41.


The William C. Whitney house on the northeast corner of 68th and Fifth, purchased, refurbished, and renovated by Stanford White for the marriage of Whitney to Edith Randolph. 14 months into the four-year project, the new Mrs. Whitney had a severe riding accident in Aiken, S.C. She died in 1899 before the renovations were completed.

Edith Randolph Whitney’s death was a terrible blow to her husband and he never quite recovered from it, dying in 1904. It changed nothing in the relationship between him and his former brother-in-law.

In 1902 Whitney’s son, Payne Whitney, who’d sided with Oliver Payne, married Helen Hay from Cleveland, Ohio. Miss Hay was the daughter of John Hay who had been private secretary to President Lincoln and later Ambassador to the Court of St. James under President McKinley. Mr. Whitney who, like his father, went to Yale, was 26. For a wedding gift, Col. Payne gave the couple the Stanford White house at 972 Fifth Avenue.

William Whitney died at age 63 in 1904. Four years later his son Payne, from whom he was estranged, and his son’s wife moved into the house at 972, with their two children, Joan and John Hay Whitney.

Payne Whitney’s only son Jock, on his wedding day in March 1942 to the former Betsey Cushing Roosevelt.

Colonel Oliver Payne died in 1917 leaving one of the largest estates, much of which went to his beloved nephew Payne Whitney, who lived a life of camaraderie and living well. He died only ten years after his uncle at age 51, leaving the largest estate ever probated (up to that time) in the United States. Among his bequests were the “forgiveness of all indebtedness” and $500,000 (about ten million in today’s currency) to his two closest friends and companions, several million to charities and the bulk to his wife, son and daughter.

Helen Hay Whitney lived in the house (part of the time) on Fifth Avenue until her death in 1944. An avid baseball fan, she spent her remaining years most of the time at the family estate Greentree in Manhasset. Upon her death, the house became the property of her son.

John Hay Whitney, known as Jock to his friends, and his sister Joan Whitney Payson, were two of the most likeable and charming people on the international social scene from the late 1920s through the 1970s. Jock Whitney is remembered as one of the greatest gentlemen of his time — generous, bright, warm and friendly.

After the Second World War, he started an investment fund, run by a friend he’d met in the War, to invest in new ideas for the men coming back from the War. He called it Adventure Capital and later dropped the “ad” to coin the now established term: venture capital. He was known for his ventures in Hollywood (Gone With the Wind), his industrious ventures, as well as being the last publisher of The New York Herald-Tribune.

Jock Whitney, Eisenhower’s Ambassador to the Court of St. James.

Like his grandfather, he was also the Ambassador to the Court of St. James (under Eisenhower). Married twice, first to a beauty who loved horses more, and finally to Betsey Cushing Roosevelt, daughter of the famous brain surgeon Harvey Cushing — first wife of FDR’s son James, to whom he remained married to the end of his life.

Jock Whitney sold the house to the French government in 1949 when it became its cultural services offices. The Payne fortune, inherited by Payne Whitney, and then his children, grew far larger than the fortune left by William C. Whitney to his children. That was partly due to the fact that Harry Payne Whitney and Gertrude Vanderbilt produced more offspring who produced more offspring. Jock Whitney produced no offspring, and his investments after the War catapulted him (and partially his sister) into the realm of what are now billions.

After his death, the fortune went to his widow and then to charity and her two Roosevelt daughters (later adopted by Whitney), and has left the arena of prominence. Joan Whitney died before her husband Charles Payson, leaving the bulk of her fortune to him, assuming that he would pass it on to their children. He remarried however and after his death (and a major lawsuit involving several wills) the fortune passed into the hands of Mr. Payson’s new wife.

Recent Posts

Subscribe

FOLLOW US