OpenAI captivated the tech world a couple of months again with a generative AI mannequin, Sora, that turns scene descriptions into unique movies — no cameras or movie crews required. However Sora has to this point been tightly gated, and the agency appears to be aiming it towards well-funded creatives like Hollywood directors — not hobbyists or small-time entrepreneurs, essentially.

Alex Mashrabov, the previous head of generative AI at Snap, sensed a chance. So he launched Higgsfield AI, an AI-powered video creation and modifying platform designed for extra tailor-made, personalised purposes.

Powered by a customized text-to-video mannequin, Higgsfield’s first app, Diffuse, can generate movies from scratch or take a selfie and generate a clip starring that particular person.

“Our target market is creators of all sorts,” Mashrabov instructed For Millionaires in an interview, “from common customers who need to create enjoyable content material with their associates to social content material creators seeking to attempt a brand new content material format to social media entrepreneurs who need their model to face out.”

Mashrabov got here to Snap by the use of AI Manufacturing facility, his earlier startup, which Snap acquired in 2020 for $166 million. Whereas at Snap, Mashrabov helped construct merchandise like AR results and filters for Snapchat, together with Cameos, in addition to Snapchat’s controversial MyAI chabot.

Higgsfield — which Mashrabov co-launched a number of months in the past with Yerzat Dulat, an AI researcher specializing in generative video — presents a curated set of pre-generated clips, a device to add reference media (i.e. photos and movies) and a immediate editor that lets customers describe the characters, actions and scenes they want to depict. Utilizing Diffuse, customers can insert themselves instantly into an AI-generated scene, or have their digital likeness mimic issues — like dance strikes — captured in different movies.

Picture Credit: Higgsfield

“Our mannequin helps extremely real looking actions and expressions,” Mashrabov stated. “We’re pioneering ‘world fashions’ for customers, which is able to permit us to construct best-in-class video technology and modifying with an excellent degree of management.”

Higgsfield isn’t the one generative video startup going face to face with OpenAI. Runway was one of many first on the scene, and its instruments proceed to enhance. There’s additionally Haiper, which has the backing of two DeepMind alums and over $13 million in enterprise money.

Mashrabov argues that Diffuse will stand out because of its mobile-first, social-forward go-to-market technique.

“By prioritizing iOS and Android apps as an alternative of desktop workflows, we allow creators to create compelling social media content material anytime and anyplace,” Mashrabov stated. “Certainly, by constructing on cell, we’re in a position to prioritize ease of use and consumer-friendly options from day one.”

Higgsfield can be operating lean. Mashrabov says that the generative fashions underpinning the platform had been developed by a 16-person group in lower than 9 months and educated on a cluster of 32 GPUs (32 GPUs would possibly sound like so much, however contemplating OpenAI makes use of tens of 1000’s, it’s not actually). And Higgsfield has solely raised $8 million so far, the majority of which got here from a latest seed funding tranche led by Menlo Ventures.

Higgsfield

Picture Credit: Higgsfield

To remain one step forward of rivals, Higgsfield plans to place the seed money towards constructing an improved video editor that’ll let customers modify characters and objects in movies, and towards coaching extra highly effective video technology fashions particularly for social media use instances. Actually, Mashrabov sees social media — and social media advertising — as Higgsfield’s precept money-making area of interest.

Whereas Diffuse is presently free to make use of, Mashrabov envisions a future the place entrepreneurs pay some kind of payment or subscription for premium options, or for quantity or large-scale campaigns.

“We consider Higgsfield unlocks an unbelievable degree of realism and content material manufacturing use instances for social media entrepreneurs,” he stated. “We continuously hear from CMOs and inventive administrators that they should optimize content material manufacturing budgets and shorten timelines whereas nonetheless delivering impactful content material. So we consider video generative AI options shall be a core resolution in serving to them to realize it.”

After all, Higgsfield isn’t immune from the broader challenges dealing with generative AI startups.

It’s well-established that generative AI fashions like the type powering Diffuse can “regurgitate” coaching knowledge. Why’s that problematic? Effectively, if the fashions had been educated on copyrighted content material with out permission or some kind of licensing settlement in place, these fashions’ customers might unwittingly generate a copyright-infringing work — exposing them to lawsuits.

Higgsfield

Picture Credit: Higgsfield

Mashrabov wouldn’t reveal the supply of Higgsfield’s coaching knowledge (aside from to say it comes from “a number of publicly out there” locations), and likewise wouldn’t say whether or not Higgsfield would retain person knowledge to coach future fashions, which could not sit proper with some enterprise clients. He did be aware that Diffuse customers can request that their knowledge be deleted at any time by means of the app.

Digital “cloning” platforms like Higgsfield are additionally ripe for abuse, because the wildfire unfold of deepfakes on social media in latest months has proven.

In an analogous vein, Higgsfield might make it simpler to steal creators’ content material. As an illustration, one want solely add a video of somebody’s choreography to generate a video of themselves performing that very same choreography.

I requested Mashrabov about what safeguards or protections Higgsfield may be utilizing to try to stop abuse, and — whereas he wouldn’t go into specifics — he claimed that the platform employs a mixture of automated and guide moderation.

“We’ve determined to step by step roll out the product and take a look at in choose markets first, in order that we will monitor the place there’s the potential for abuse and evolve the product as crucial,” Mashrabov added.

We’ll have to attend and see how nicely that works in apply.