To quote from Wikipedia:
While most railways use the standard gauge of 1435 mm, some countries, especially Spain and the former member states of the Soviet Union have widespread broad gauge tracks (1,520 mm). Likewise, electrification of lines varies between countries. 15 kV AC has been used in Germany, Austria, Switzerland, Norway and Sweden since 1912, while the Netherlands uses 1500 V DC, France uses 1500 V DC and 25 kV AC, and so on. All this makes the construction of truly pan-European vehicles a challenging task […]
It probably would be prohibitively expensive to unify everything, but rational decisions need to be taken, and that can’t happen on a national basis. For instance, Danish politicians are currently discussing whether to electrify anything other than the route from Copenhagen over Funen to the German border; this is obviously a no-brainer on a European scale, but to a small country the cost can still be prohibitive.
At the moment, efforts are concentrated on the high-speed lines, but wouldn’t it be more efficient to hand over the entire railway networks to the EU? I’m not saying EU politicians should necessarily unify everything, but they could at least change situations where one country is the odd man out.
Actually running trains could still be handled/outsourced by the individual countries, but it would be so much easier to create attractive franchises if a train in most cases could run directly from any point in the EU to another other point, and if trains purchased for use in other country could just be used in another.