Earlier this week, The Wall Street<\/em> Journal<\/em> reported that AI companies were running into a wall when it comes to gathering high-quality training data. Today, The New York Times<\/em> detailed some of the ways companies have dealt with this. Unsurprisingly, it involves doing things that fall into the hazy gray area of AI copyright law.<\/p>\n<\/div>\n The story opens on OpenAI which, desperate for training data, reportedly developed its Whisper audio transcription model to get over the hump, transcribing over a million hours of YouTube videos to train GPT-4, its most advanced large language model. That\u2019s according to The New York Times<\/em>, which reports that the company knew this was legally questionable but believed it to be fair use. OpenAI president Greg Brockman was personally involved in collecting videos that were used, the Times<\/em> writes. <\/p>\n<\/div>\n OpenAI spokesperson Lindsay Held told The Verge <\/em>in an email that the company curates \u201cunique\u201d datasets for each of its models to \u201chelp their understanding of the world\u201d and maintain its global research competitiveness. Held added that the company uses \u201cnumerous sources including publicly available data and partnerships for non-public data,\u201d and that it\u2019s looking into generating its own synthetic data. <\/p>\n<\/div>\n The Times<\/em> article says that the company exhausted supplies of useful data in 2021, and discussed transcribing YouTube videos, podcasts, and audiobooks after blowing through other resources. By then, it had trained its models on data that included computer code from Github, chess move databases, and schoolwork content from Quizlet.<\/p>\n<\/div>\n Google spokesperson Matt Bryant told The Verge<\/em> in an email the company has \u201cseen unconfirmed reports\u201d of OpenAI\u2019s activity, adding that \u201cboth our robots.txt files and Terms of Service prohibit unauthorized scraping or downloading of YouTube content,\u201d echoing the company\u2019s terms of use. YouTube CEO Neal Mohan said similar things about the possibility that OpenAI used YouTube to train its Sora video-generating model this week. Bryant said Google takes \u201ctechnical and legal measures\u201d to prevent such unauthorized use \u201cwhen we have a clear legal or technical basis to do so.\u201d<\/p>\n<\/div>\n Google also gathered transcripts from YouTube, according to the Times\u2019 <\/em>sources. Bryant said that the company has trained its models \u201con some YouTube content, in accordance with our agreements with YouTube creators.\u201d <\/p>\n<\/div>\n The Times <\/em>writes that Google\u2019s legal department asked the company\u2019s privacy team to tweak its policy language to expand what it could do with consumer data, such as its office tools like Google Docs. The new policy was reportedly intentionally released on July 1st to take advantage of the distraction of the Independence Day holiday weekend.<\/p>\n<\/div>\n Meta likewise bumped against the limits of good training data availability, and in recordings the Times<\/em> heard, its AI team discussed its unpermitted use of copyrighted works while working to catch up to OpenAI. The company, after going through \u201calmost available English-language book, essay, poem and news article on the internet,\u201d apparently considered taking steps like paying for book licenses or even buying a large publisher outright. It was also apparently limited in the ways it could use consumer data by privacy-focused changes it made in the wake of the Cambridge Analytica scandal.<\/p>\n<\/div>\n Google, OpenAI, and the broader AI training world are wrestling with quickly-evaporating training data for their models, which get better the more data they absorb. The Journal<\/em> wrote this week that companies may outpace new content by 2028. <\/p>\n<\/div>\n Possible solutions to that problem mentioned by the Journal <\/em>on Monday include training models on \u201csynthetic\u201d data created by their own models or so-called \u201ccurriculum learning,\u201d which involves feeding models high-quality data in an ordered fashion in hopes that they can use make \u201csmarter connections between concepts\u201d using far less information, but neither approach is proven, yet. But the companies\u2019 other option is using whatever they can find, whether they have permission or not, and based on multiple lawsuits filed in the last year or so, that way is, let\u2019s say, more than a little fraught.<\/p>\n<\/div>\n<\/div>\n[ad_2]\n","protected":false},"excerpt":{"rendered":" [ad_1] Earlier this week, The Wall Street Journal reported that AI companies were running into a wall when it comes to gathering high-quality training data. Today, The New York Times detailed some of the ways companies have dealt with this. Unsurprisingly, it involves doing things that fall into the hazy gray area of AI copyright …<\/p>\n","protected":false},"author":1,"featured_media":35119,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[2],"tags":[],"_links":{"self":[{"href":"https:\/\/farratanews.online\/wp-json\/wp\/v2\/posts\/35118"}],"collection":[{"href":"https:\/\/farratanews.online\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/farratanews.online\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/farratanews.online\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/farratanews.online\/wp-json\/wp\/v2\/comments?post=35118"}],"version-history":[{"count":0,"href":"https:\/\/farratanews.online\/wp-json\/wp\/v2\/posts\/35118\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/farratanews.online\/wp-json\/wp\/v2\/media\/35119"}],"wp:attachment":[{"href":"https:\/\/farratanews.online\/wp-json\/wp\/v2\/media?parent=35118"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/farratanews.online\/wp-json\/wp\/v2\/categories?post=35118"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/farratanews.online\/wp-json\/wp\/v2\/tags?post=35118"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}