Lewis Bayly (died 1631), grandfather of the first Baronet, was Bishop of Bangor . His son Nicholas Bayly , father of the first Baronet, was a member of the Irish House of Commons for Newry. The Very Reverend Edward Bayly, younger son of the first Baronet, was Archdeacon of Dublin. Lambert Bayly, younger son of the first Baronet, was the father of the Very Reverend John Bayly, Dean of Killaloe, who was the grandfather of (1) Paget Bayly, a major-general in the Army, and (2) John Bayly, a general in the Army. Charles Bayly, younger son of the first Baronet, was a captain in the Royal Navy. Paget Bayly, younger son of the second Baronet, was also a captain in the Royal Navy.  Nicholas Bayly , younger son of the second Baronet, was Member of Parliament for Anglesey.
In December 1828, Paget addressed a letter to the Roman Catholic primate of Ireland stating his belief in the need for Catholic emancipation , which led to his recall by the government; on the formation of Earl Grey's administration in November 1830, he again became lord-lieutenant of Ireland.   In this capacity he introduced state-aided education for 400,000 children.  In July 1833, the ministry resigned over the Irish question. Paget spent the following thirteen years out of office, then joined Lord John Russell 's administration as Master-General of the Ordnance in July 1846.  He was promoted to field-marshal on 9 November 1846  and, having been appointed Lord Lieutenant of Staffordshire on 31 January 1849,  he finally retired from the Government in March 1852.