Chromosomal sex is determined by the specific combination of sex chromosomes an individual inherits, typically XX for females and XY for males in humans. This genetic determination plays a crucial role in the development of primary and secondary sexual characteristics through the regulation of sex-specific gene expression.