Hello!
Currently I play as Serco.
When ever I enter the Deneb sector to fight the border skirmish mission, all of the ships are marked red on the radar, and when I try to select nearest enemy, I select the nearest ship (of any nation) as if all the ships, regardless of whether they are Serco or Itani, are treated as enemies.

This makes it a little hard to fight, as it is impssible to see if that ship closing in on you from the rear is friendly or not. :(

That's (unfortunately) a known issue with the way the current faction standings and friend/foe identification work on radar. Both of those things are being completely reworked according to recent news from the developers.

The basic issue:
A target on your radar is marked hostile (red) if any of the following apply:
1. Target ship has Hated or lower standing with your nation
2. You have Hated or lower standing with the other ship's nation/faction
3. Target ship has Hated or lower standing with the nation/faction that controls the current sector.

Deneb B12 is Itani controlled and monitored, so the majority of Serco (especially those participating in Border Skirmish and Border Patrol, and all Serco NPCs) are marked red on everyone's radar, regardless of their standing with each other.