Comedian Matt Richardson, 22, was announced last week as her new sidekick, sparking speculation that Caroline, 33, would once again find herself falling for a younger up-and-coming star.
The presenter notoriously dated Harry Styles for several months in 2011 when the One Direction singer was just 17.
And last year the audience watched agog as ITV2 spin-off turned into a will-they-won’t-they saga about Caroline and then co-host, Olly Murs, now 29.
She and the former X Factor runner-up - who bowed out of this year’s show because of his music commitments - denied anything ever happened between them.
But the chemistry seemed sizzlingly obvious on-screen, and they were also spotted cuddling backstage.
So Caroline is resigned to speculation about her relationship with newbie Matt - who at a stand-up gig in Brighton last April poked fun at Caroline’s fling with Harry.
After joking about how the boyband singer only looked about 12 years old, cheeky Matt responded to a woman giggling in the audience by saying: “One woman orgasmed there, obviously a Harry Styles fan. Could have been Caroline Flack.”
Ladbrokes are already offering odds of 2-1 for Caroline and Matt to hook up by the end of the year, but she insists NOTHING will happen.
She says: “There probably will be stories, particularly because of his age! But I don’t fancy him.
“He is more like my naughty younger brother and I’m going to have to keep him in line - but in a good way, not in a dominatrix way!
“He has already told me he does really badly with women, so I think I’m just going to completely take the mickey out of him all the time.
“He has already been texting me asking me for advice about what he should do with girls.”
And Caroline - who begins filming with Matt today in Glasgow at the first auditions for the new X Factor series which will screen in August - insisted she is already all loved-up.
She said: “I am dating at the moment. That is all I am saying! It could be a woman. No, it is a man. I am very happy at the moment. Life is really good at the moment and I am looking forward to the rest of the year.”
Xtra Factor co-star Matt Richardson mocked Caroline Flack's taste in men.
Caroline recently sparked rumours that she had secretly got ENGAGED to her mystery man after posting a picture of herself online with a ring on the all-important finger.
But she admits it was just a joke, explaining: “My friend had just got married and her ring was so beautiful she let me try it on but it just looked weird on my finger.
“I will probably never get married so it was my one chance!”
Caroline has spent the past few weeks furiously getting herself into peak condition for the new Xtra Factor series.
She said: “I am ready for it now. I have got myself ready - I have been eating healthily, I’ve been on holiday so I’ve been getting back in shape. I have been working out every day.
“I do feel that pressure of being on TV and having to look good, definitely. It is easy when you are off work to have a massive blow-out and eat loads and not work out.”
And she added: “Xtra Factor is hard work - it is full-on from now until the end of the year. But I think it’s going to be a really good year." But she suspects that the show will have a very different vibe this time because of comic Matt.
She said: “There are going to be quite a lot of changes. Matt is a completely different character to Olly. It might be that we do some stuff separately rather than always being together.
“But it will still be lots of fun. He is a comedian so he will want to do things differently as well. He is a very nice guy, he is very likeable.
“We said we are not going to replace Olly and that we wanted someone completely different. Matt is almost like the opposite of Olly.
“He brings a completely different feel to it. I am really excited about working with him. He is very funny.
“We have really good chemistry together and I helped make the decision.
“It wasn’t a case of just choosing someone and putting them in, it was about how we worked together.
“There were quite a few people interviewed for the job and then we screen tested the people we felt were right. He brought something to it which we hadn’t even thought about.”
And Caroline said her new co-host has also been given the thumbs up by supremo Simon Cowell.
She said: “He really likes Matt. He is really excited about the show.
“This will be my third year now. I am the longest-serving female host on The Xtra Factor.
“I could be the female Louis Walsh!”
Providing X Factor News, Information And Reviews Online.
Tonight our fabulous Finalists got into the swing of all things brilliantly British! From beautiful ballads, to heavenly harmonies our singing superstars put on some perfectly patriotic performances.