Charles Bernard Spencer was born in Madras, India, in November 1909. He came to England at the age of two and was brought up by relatives and guardians. He attended Marlborough School, where he met Louis MacNeice and John Betjeman. After graduating in 1932 from Corpus Christi College, Oxford, with a degree in Greats, he worked in a number of teaching and other jobs before joining the British Council in 1940.

In August 1936, he married Nora Gibbs, but they were to be separated between 1940 and 1945, she in London, he in Greece and then Cairo, thanks to the vicissitudes of war. Together on his posting to Palermo, Sicily, in 1946, she developed TB and died in Rome the following June. Spencer himself then underwent a painful cure for the same disease in Switzerland. There followed postings to Turin, Madrid, Athens, Ankara, Madrid again, and finally Vienna. He married Anne Marjoribanks in September 1961 and their son Piers was born in February 1963. His second collection With Luck Lasting was a Poetry Book Society Recommendation that summer.

While on holiday in Venice, Spencer began to suffer from an undiagnosed illness. Back in Vienna, feverish and with a high temperature, he walked out of the clinic where he was under observation and would be discovered with head injuries beside suburban railway lines in the early hours of 11 September 1963. The exact circumstances of his death have never been fully explained.

Roger Bowen's edition of Spencer's Collected Poems was published by OUP in 1981. His Complete Poetry, Translations and Selected Prose, edited by Peter Robinson, was published by Bloodaxe Books in 2011.