The US said its planes had halted the attack in Deir al-Zour when informed of the Syrian presence.
A spokesman for the US administration expressed “regret” for the “unintentional loss of life”.
An emergency meeting of the United Nations Security Council, convened by Russia, is discussing the crisis.
The Russians earlier said the current ceasefire in Syria was in danger of collapse and the US would be to blame.
The cessation of hostilities does not include attacks by the US on IS or other jihadist groups.
The US Central Command statement said the coalition believed it was attacking positions of so-called Islamic State and the raids were “halted immediately when coalition officials were informed by Russian officials that it was possible the personnel and vehicles targeted were part of the Syrian military”.
It said the “Combined Air Operations Center had earlier informed Russian counterparts of the upcoming strike”.
It added: “Syria is a complex situation with various military forces and militias in close proximity, but coalition forces would not intentionally strike a known Syrian military unit. The coalition will review this strike and the circumstances surrounding it to see if any lessons can be learned.”
Russia’s defence ministry earlier said that if the US air strikes did turn out to be an error, it would be because of Washington’s refusal to co-ordinate military action with Moscow.
Only if the current ceasefire – which began on Monday – holds for seven days, will the US and Russia begin co-ordinated action against the Jabhat Fateh al-Sham group, which was previously known as the al-Nusra Front, and IS.
The Russian defence ministry quoted a statement by Syrian army general command as saying that the four coalition air strikes on Syrian troops had allowed IS to advance.
The Russian foreign ministry said the attack had jeopardised the US-Russia agreement on Syria.
The Syrian statement said that the air strikes were “conclusive evidence” that the US and its allies supported the jihadist group.
The Syrian Observatory for Human Rights monitoring group put the death toll at at least 80.
There have been no confirmed cases of US air strikes targeting Syrian troops. Last December, Syria accused the coalition of attacking a government army camp in Deir al-Zour but the US denied it.