When a quantum dot is prepared in its biexciton state, i.e., the state containing two electron-hole pairs, the subsequent cascaded emission of photons can result in an polarization-entangled photon state. The quality of the entanglement is encoded in the concurrence, which in the ideal case is 100%. In this paper we show that the excitation process can influence the degree of concurrence. When the biexciton state is prepared via a two-photon excitation, the excitation process itself induces a which-path information, which reduces the achievable concurrence. For a typical pulse length of about 10 ps, the concurrence is reduced to about 97.5%, which agrees well with the measurable record values. The paper gives a fundamental understanding of the process, such that now new strategies can be developed to reach the goal of having perfectly entangled photons from quantum dots.