His son Mikołaj Radziwiłł “The Old” extended the Biržai estate even further. Jerzy Radziwiłł (1480-1541), Mikołaj's son and the Great Hetman of the Great Duchy of Lithuania, inherited Biržai. Jerzy additionally expanded the Biržai estate by purchasing land from nobles in Upytė. At the beginning of the 16th century, there were 503 households in the Biržai lands. This property was one of the larger estates belonging to Jerzy Radziwiłł. He was the first to expand it towards the northern border along the Nemunėlis River.
His only son, Mikołaj Radziwiłł “The Red” (1512-1584) inherited the estate. Mikołaj was granted the title Reichsfurst (Prince) of the Holy Roman Empire, thus renaming this area the Duchy of Biržai-Dubingiai.
In historical records and archives Biržai is mentioned as a township, the nearest castle at the time being located in Papilys. Mikołaj Radziwiłł “The Red” was the first to accommodate Tatars and Germans in his Biržai lands. In 1570, there were 12 German households in Biržai, covering 521 voloks (late medieval unit of land measurement). The first Tatars were given land in Niutanai and Aspariškiai.
Chronicles from 1561 describe the estate which had four barns, a brewery, a bakery, a kitchen and three manors: Joniškėlis, Ageniškis and Papilys. In 1584, when Mikołaj Radziwiłł “The Red” passed away, his sons divided the lands between them. Biržai, Papilys, Ageniškis and Joniškelis became the property of Krzysztof Radziwiłł “The Thunderbolt” (1547-1603). He purchased Naradava Manor in 1584 and also bought large farmstead in Cypiškiai. In the 1590s, Krzysztof Radziwiłł focused lot of his attention and resources to Biržai and turnded it into main residence of the Radziwiłł.
In 1589, the construction of a bastion-type fortress was completed here. Same year, on March 9th, the town of Biržai was granted Magdeburg Rights. Krzysztof's last big acquisitions were the lands of Kiemenai, in 1598, and Saločiai from the noble Komarovsky in 1601. The Duchy of Biržai was turned into the Radziwiłł family’s representative and defensive Biržai-Dubingiai property.
After Krzysztof Radziwiłł “The Thunderbolt” died, his will was carried out. It was Krzysztof Radziwiłł (1585-1640), who inherited Biržai, Nemunėlio Radviliškis, Papilys, Astravas, Salamiestis, Ageniškis, Parovėja, Naradava and Saločiai. The Biržai Duchy remained unchanged until the second half of the 17th century. Krzysztof Radziwiłł focused on creating an administrative structure for the duchy.
He died in 1640 and his son, Janusz Radziwiłł (1612-1655), inherited the duchy. Alike his father, Janusz, dedicated lot of his time to it. In the times of Krzysztof Radziwiłł and Janusz Radziwiłł, the Biržai duchy had one town, four townships and between 250 and 300 villages and farmsteads. In the first half of the 17th century, the Radziwiłł did not expand their territory. That was done later, during the times of Bogusław Radziwiłł (1620-1669).
In the middle of the 17th century, the Duchy of Biržai consisted of five domains:
1. Biržai (600 voloks): the town of Biržai, the manor and rural subdivisions, and 60 villages and large farmsteads.
2. Papilys (more than 400 voloks, and 40 villages and farmsteads).
3. Nemunėlio Radviliškis (around 250 voloks and 33 villages and farmsteads).
4. Salamiestis (224 voloks, a township, rural subdivisions and a forest of around 119 voloks).
5. Saločiai (around 150 voloks).
Biržai town was strategic, religious and representative center of the Duchy of Biržai. The inhabited territory of the duchy covered almost 36 000 hectares, and in addition to that, there were huge forests, water basins and meadows. The Duchy of Biržai was the Radziwiłł's pride and it could compete with any estate of the Polish-Lithuanian nobleman of that time.
The Grand Duke of Lithuania and King of Poland Sigismund III Vasa and later on his descendants would not renounce their claims to the Swedish throne, which meant war for Lithuania and Poland. The war with Sweden lasted many decades (1600-1635 and 1655-1660) and in 1625 the Swedes turned Biržai Castle into ruins.
Krzysztof Radziwiłł initiated reconstruction works on the Biržai fortress in 1637. It did not only include restoration, but also the change from Italian to Dutch style. This meant that the bastions and ramparts surrounding the fortress become its main features. Residential palace and the grounds were also reconstructed.
In 1640, Biržai was inherited by the future Grand Hetman of Lithuania, son of Krzysztof Radziwiłł, Janusz Radziwiłł (1612-1655), who continued the reconstruction of the castle.
In autumn 1655, the Biržai fortress changed hands again and became Swedish.
In 1659, Janusz Radziwiłł’s cousin, Duke Bogusław Radziwiłł (1620-1669) took back the castle from the Swedes and it was once again in the possession of the Radziwiłł family. Although he resided in Königsberg (Karaliaučius), he initiated the castle's reconstruction, which has been hindered by the wars. In 1662, the technical plans of the fortress were drawn up. Unfortunately, the reconstruction came to a halt in 1669 because of the duke's death. Bogusław Radziwiłł was the last male offspring of the Biržai-Dubingiai branch of the Radziwiłł family.
In his testament, he left the fortress to his daughter Ludwika Karolina Radziwiłł (1667-1695). She restarted the reconstruction process in 1671 and work continued until 1682. This fortress was considerably larger than the first one. The palace, fortifications and 21 buildings made it a large defensive complex.
On February 26, 1701, during the Great Northern War (1700-1721), Russian Tsar Peter I and King of the Polish-Lithuanian Commonwealth Augustus II signed a treaty against the Swedes in Biržai. In August that same year, the Swedes occupied Biržai and kept hold of it until 1703.
On September 14, 1704, the army of Swedish General Adam Ludwig Lewenhaupt took over the fortress once again. The Swedes blew up the palace and the fortress buildings before retreating.