The Rev. Peter Morales, president of the Unitarian Universalist Association, attended the rally and was among a group that accompanied Arpaio on a tour of the tent city.
In 2010, Morales was arrested at a protest in Phoenix over Arizona's immigration law.
Rachel Walden, who works at the Unitarian Universalist Association headquarters in Boston, said the group also gathered Saturday to protest the immigration law.
"We wanted to stand in solidarity with the immigration groups," she said. "We just wanted to be there to have a big presence and say this is wrong."
Arpaio is a national political fixture who built his reputation on jailing inmates in tents during Phoenix's triple-digit summer heat, dressing inmates in pink underwear, selling himself to voters as unceasingly tough on crime, and pushing the bounds of how far local police can go to confront illegal immigration.
The U.S. Justice Department has accused the sheriff's office of racially profiling Latinos in its immigration patrols.
The sheriff has said the government's investigation of his immigration patrols amounted to a politically motivated attack by the Obama administration and has denied allegations of systematic discriminatory policing.
The jail complex in west Phoenix was the site of a January 2010 protest that drew 10,000 immigrant rights advocates and was marked by a clash between a small group of protesters and police officers.
In 1997, a report by Amnesty International found the structure fell short of being an "adequate or humane alternative to housing inmates in suitable jail facilities", according to the Phoenix New Times.
Arpaio has repeatedly said he doesn't see any problems with housing inmates in tents and often points out that some members of the U.S. military live in tents.
The county's jails were closed to visitors on Saturday because of the rally.
Intellpuke: You can read this article by Guardian correspondent Matt William, with reporting by various news agencies, in context here: www.guardian.co.uk/world/2012/jun/24/thousands-protest-sheriff-arpaio-jail