Many of these 19th-century structures are located along New York State Route 25A, otherwise known as North Country Road. Among them is Saint James Episcopal Church, the source of the hamlet's name during the 1850s. In 1873, St. James Rail Road Station was established. Although small, the historic station-house is the second-oldest on the Long Island Rail Road.
According to the United States Census Bureau, the CDP has a total area of 4.5 square miles (12 km2), all land. To the north of the hamlet lies the Village of Head of the Harbor and Stony Brook Harbor.
St. James is situated in the eastern part of the town of Smithtown, east of the hamlet of Smithtown. Immediately due east of St. James sits the main campus of Stony Brook University, a major public research university.
As of the census[7] of 2000, there were 13,268 people, 4,555 households, and 3,466 families residing in the CDP. The population density was 2,924.6 inhabitants per square mile (1,129.2/km2). There were 4,674 housing units at an average density of 1,030.3 per square mile (397.8/km2). The racial makeup of the CDP was 97.36% White, 0.27% African American, 0.05% Native American, 1.24% Asian, 0.01% Pacific Islander, 0.55% from other races, and 0.52% from two or more races. Hispanic or Latino of any race were 3.45% of the population.
There were 4,555 households, out of which 36.6% had children under the age of 18 living with them, 66.5% were married couples living together, 7.2% had a female householder with no husband present, and 23.9% were non-families. 20.6% of all households were made up of individuals, and 13.3% had someone living alone who was 65 years of age or older. The average household size was 2.81 and the average family size was 3.27.
In the CDP, the population was spread out, with 25.9% under the age of 18, 5.0% from 18 to 24, 29.8% from 25 to 44, 21.2% from 45 to 64, and 18.1% who were 65 years of age or older. The median age was 39 years. For every 100 females, there were 88.4 males. For every 100 females age 18 and over, there were 82.6 males.
The median income for a household in the CDP was $71,144, and the median income for a family was $82,291. Males had a median income of $59,018 versus $38,103 for females. The per capita income for the CDP was $29,643. About 1.6% of families and 2.6% of the population were below the poverty line, including 1.8% of those under age 18 and 2.2% of those age 65 or over.