Police say the attack was staged, which Smollett denies.
Olabinjo Osundairo and Abimbola Osundairo say their reputations have been damaged as a result of the claims.
Smollett's lawyers Mark Geragos and Tina Glandian described the lawsuit as "comical" and "ridiculous".
The brothers, known as Ola and Abel, said in a statement: "We have sat back and watched lie after lie being fabricated about us in the media only so one big lie can continue to have life.
"These lies are destroying our character and reputation in our personal and professional lives."
Smollett's legal team said: "At first we thought this comical legal document was a parody.
"Instead this so-called lawsuit by the brothers is more of their lawyer-driven nonsense, and a desperate attempt for them to stay relevant and further profit from an attack they admit they perpetrated.
"While we know this ridiculous lawsuit will soon be dismissed because it lacks any legal footing, we look forward to exposing the fraud the Osundairo brothers and their attorneys have committed on the public."
The Jussie Smollett case has been a complicated one to keep up with.
- How the Jussie Smollett drama unfolded
On 14 February the Osundairo brothers were arrested by Chicago police and questioned. They had worked as extras on Empire, Smollett's show, sometimes going to the gym with the actor.
A few days later the brothers were released by police without charge and, on 20 February, Jussie Smollett was charged with filing a false police report.
Smollett was arrested and police say he paid the Osundairo brothers to stage the attack.
On 1 March, the Osundairos said they "regret" their involvement in the incident.
Two weeks later, Smollett pleaded not guilty to 16 counts of disorderly conduct - and 12 days after that all charges against him were controversially dropped.
This is when lawyers for the Osundairo brothers say Smollett's attorneys "doubled down" on "untrue" statements that the brothers were behind the "criminally homophobic, racist, and violent attack" against Smollett.
They claim defamatory statements were made in interviews by one of Smollett's lawyers, Tina Glandian, the day after charges against him were dropped, and then repeated widely across the media.
Olabinjo and Abimbola Osundairo are seeking financial compensation for the "harm" done to them as a result of the statements, which they say have caused "irreparable financial damage".
Lawyers for the brothers say they've suffered "significant emotional distress and feel unsafe and alienated in their local Chicago community" as a result of the comments.
"Their lives have been forever changed and damaged by the words and acts of others," said lawyer Gloria Schmidt.
"Let me make one thing perfectly clear, the brothers have done nothing but tell the truth to CPD and to the Grand Jury."