Magid Magid, 30, was wearing a baseball cap and a T-shirt with swearing and an anti-fascist slogan on it when he was asked to leave.
The former Lord Mayor of Sheffield was elected as one of six MEPs for the Yorkshire and Humber region in May.
The European Parliament said no member of staff was involved in the incident.
Mr Magid said he did not know who the person was who asked him to leave, although he believed that individual to be an official.
He said the person asked if he was lost and then suggested he leave.
He added: “I make people feel uncomfortable, people don’t know how to react.”
In a tweet, he said: “I know I’m visibly different. I don’t have the privilege to hide my identity. I’m BLACK & my name is Magid.
“I don’t intend to try fit in. Get used to it!”
He said the exchange said a lot about what people thought politicians were supposed to look like, and he did not leave the building.
A spokeswoman for the European Parliament said: “We investigated the matter immediately after our attention was brought to it and can safely say that no member of Parliament staff was involved.”
Mr Magid was attending the opening of the new five-year session of the parliament, though the length of the UK’s involvement remains in doubt.
UK MEPs may sit in the parliament until the country formally leaves the EU.
Mr Magid came to Sheffield aged five from an Ethiopian refugee camp “to find a better life”.
He was a contestant on Channel 4’s reality show Hunted, and was elected Broomhill and Sharrow Vale Green councillor in 2016 and became the city’s youngest Lord Mayor in 2018.
He went on to cause controversy when, in July 2018, he “banned” visiting US President Donald Trump from Sheffield.