Our Lady of Mercy Church was established in 1927, as the Chapel of the Little Flower (located at Continental Avenue and Kessel Street), a mission church of nearby Our Lady Queen of Martyrs Church. From 1927 to 1930, the Chapel of the Little Flower was served by clergy from Our Lady Queen of Martyrs Parish. Father James B. Collins was the first chaplain.

On Christmas Day 1929, a new wing of the Chapel was opened to accommodate the increasing number of parishioners and on October 6, 1930, the Chapel of the Little Flower was rededicated under the title of Our Lady of Mercy Parish. Father Edward P. Costello was named the first pastor of the parish and immediately began to plan for the building of a church adequate to meet the needs of the growing congregation..

On September 26, 1937, Archbishop Thomas E. Molloy, Bishop of Brooklyn, dedicated the newly constructed church building of Our Lady of Mercy.
Father Costello died on January 6, 1939 and Father Christopher T. Molloy was appointed the second pastor of the parish shortly thereafter. During his pastorate, from 1939 to 1963, Father Molloy began planning for the building of a school and convent and, at his death in February 1963, he left the parish in sound financial condition to bring his plans to fruition.
Father Molloy¹s successor, Monsignor Matthew E. Hickey, was named the third pastor of the parish in June 1963. The building of a school and convent was undertaken by Monsignor Hickey and on May 14, 1967, Archbishop Bryan J. McEntagart, Bishop of Brooklyn, officially dedicated the new buildings.

In 1967, upon the sudden death of Monsignor Hickey, Father John J. Johnston was appointed the fourth pastor of Our Lady of Mercy. Several years later, the church building was renovated to better conform to the liturgical changes which came about as a result of Vatican Council II.
It was rededicated on May 24, 1975 by Bishop Francis J. Mugavero, Bishop of Brooklyn. Upon Monsignor Johnston's retirement, the current pastor, Monsignor Gerald J. Langelier, became the fifth pastor of the parish on June 9, 1984.

In honor of the church building¹s 50th Anniversary, it was refurbished to its present liturgical setting in 1987 and rededicated by Bishop Francis J. Mugavero, Bishop of Brooklyn, on January 31, 1988.
Our Lady of Mercy Parish currently serves approximately 1300 families in the Forest Hills area.