Seferovic's winner capped a remarkable comeback by the top seeds in the group as fellow substitute Admer Mehmidi had taken just 121 seconds to make his mark as he levelled the game in the 48th minute.
Enner Valencia had given Ecuador the lead midway through the first-half as he scored for the fifth successive game for his country.
Victory was Switzerland's first over South American opposition in the World Cup finals having lost their previous four and the win had even veteran coach Ottmar Hitzfeld jumping up and down for joy in what is his farewell tournament.
Hitzfeld, who guided Borussia Dortmund and Bayern Munich to Champions League glory, said the win would do wonders for the morale of his team and set them up nicely for further games against France and Honduras.
"I really thought this move was delightful," said the 68-year-old German, who will be mindful that in 2010 the Swiss opened with a win over eventual champions Spain but ended up going out in the first round.
"We reacted intuitively and knew how to take advantage of the Ecuadorean mistake. It is marvellous to win in the last minute as it releases positive energy and is very good for the morale of the team."
Ecuador coach Reinaldo Rueda said he had been in the game for long enough to not be shocked by losing in that manner.
"Football is like that, you can lose in the last minute," said the Colombian, who will face the team he took to the 2010 finals Honduras later in the group stage.
"We lost our heads a bit in the final minute but we know that. Of course it is a tough hit to our morale but my boys have a lot of guts and are able to bounce back. It isn't as if the Swiss thrashed us."
Ecuador had started the brightest and it was little surprise when they went ahead in the 22nd minute as former captain Walter Ayovi whipped in a superb freekick from the left and the unmarked Enner Valencia rose to send his header past Diego Benaglio and into the net.
The Swiss were finding it hard to settle and it wasn't until the 35th minute they managed a shot on target as captain Goekhan Inler worked some space and unleashed a vicious low shot that Ecuador goalkeeper Alexander Dominguez did well to get down to and turn away for a corner.
However, Dominguez was all at sea for the Swiss equaliser losing himself behind two defenders as the corner came in from Ricardo Rodriguez and Mehmedi, who scored 13 goals for Freiburg in the Bundesliga last season, rose above Carlos Gruezo to head home for his second goal for his country.
The lively Enner Valencia then produced a lovely piece of skill and unleashed a fine effort from outside the box which went narrowly wide of the Swiss goal.
Both teams were not content to settle for a draw even as the match entered time added on and it was Ecuador who looked like they had grabbed the three points when Michael Arroyo was shaping up to shoot inside the area only for a superb tackle by Valon Behrami to dispossess him.
Behrami showing the hard-headed experience of a man appearing in his third World Cup finals, then pushed forward and although he was subject to a crunching tackle in the middle of the pitch the referee played a brilliant advantage in waving play on.
The ball was worked out to the left flank and when it was whipped in Seferovic was on hand to tuck the ball inside the near post and send the Swiss contingent wild. (AFP)