Aj$6dZddlmZddlZddlZddlZddlZddlmZm Z ddl m Z ddl Z ddlmZddlmZmZdd lmZmZmZmZdd lmZmZmZej8d Zd Zd ZeGddZ ddZ!dddZ"ddZ#ddZ$dddZ%d dZ&d!dZ'd"dZ(d#dZ)y)$aTrack discovery for m x n evaluation: which audio stream is the original, which streams are dubs, and what language each one really is. Container language tags are unreliable (the 3 Idiots original is tagged `eng` but is Hindi), so every candidate stream gets Whisper language-ID on its speech. Streams that are a bit-for-bit copy of the original audio (dub files often carry the source track along) are skipped. ) annotationsN) dataclassfield)Path) fftconvolve)audiospeech)Langbase_ofresolve same_language)Probe extract_audio probe_cacheddubeval?g?ceZdZUded<ded<ded<ded<ded <ded <d Zd ed <dZded<ed dZded<ed dZded<ed dZ ded<e ddZ e ddZ e d dZ d!dZd"dZd#dZy )$TrackstrrolerpathintindexdictstreamdirlidNz Lang | Nonelangdetected lang_sourceF)defaultreprznp.ndarray | None_xz dict | None_asrz tuple | None_stemsc.|jjSN)rnameselfs 2/root/repos/pocs/video-evals/src/dubeval/tracks.pyslugz Track.slug,sxx}}c |jdz S)Na16k.wav)rr*s r,wavz Track.wav0sxx*$$r.cz|j$tj|j|_|jSr()r$r loadr1r*s r,xzTrack.x4s( 77?kk$((+DGwwr.c |j_tj|j|j|j dz |j |j|j||_|jS)Nzasr.json) r%r transcriber1rrasr_cloud_modelasr_local_modelthreads)r+cfgwarningss r,asrz Track.asr:sZ 99 ))$((DIItxx*?TVYViVi*-*=*=s{{HVDIyyr.c|j|j|jdf|_|r t|j|j|j dz d}t j||j dz |\}}tj|tj|df|_|jS|jS#t$r6}|jd|jd|Yd}~|jSd}~wwxYw) zI(vocals, background, separated). Without separation both are the raw mix.NFza44k.waviDsepTzdemucs separation failed for z, using raw mix: ) r&r4rrrrr separater r3 Exceptionappendr-)r+enabledr9r;wav44vbes r,stemsz Track.stems@s ;; 6646651DKe)$))TZZJAVX]^E >>%E1A7KDAq#);;q>6;;q>4"HDK{{t{{!eOO&CDII;N_`a_b$cdd{{esBC D !DD ct|j|j|jj d|j |j dn|j j|jdS)Nr)file audio_index container_tagr languagelanguage_source) rrrrgetrrto_dictr!r*s r,describezTrack.describeMs_DIItzzTXT_T_TcTcdjTk HH$)):K$QUQZQZQbQbQd#'#3#35 5r.)returnr)rQr)rQ np.ndarray)r; list[str]rQr)rBboolr9rr;rSrQz#tuple[np.ndarray, np.ndarray, bool])rQr)__name__ __module__ __qualname____annotations__rr!rr$r%r&propertyr-r1r4r<rGrPr.r,rrs I J J L I ID+!K!!$U;B;d7D+7 E:FL: %%  5r.rctjt|jj j dd}t jdd|jddd|d|S)Nz[^A-Za-z0-9._-]+_0-z.a) hashlibsha1rr encode hexdigestresubstem)rrhs r,_slugrhSse S(//12<<>rBAff(#tyy9#2> ?q2eW MMr.c  tj|}gd}}|D]\\}}|j|t|tjzt|tjz|||z z }||k\s\n|dkrddgdStj ||} | j tj|d\} } } | tt| d| ddD cgc]\} }| tt|dgc}} dScc}} w)z^Whisper language-ID over the first `max_speech` seconds of VAD speech (music/silence removed).gg@N)rprobtop)r language_detection_segments) r vad_segmentsrArSR whisper_modeldetect_languagenp concatenateroundfloat)r4modelr9 max_speechsegschunkstotalsrFmcoderjallpcps r,rrrrXs    q !DEF1 aA M*3q699}+=>? Q J     s{c"55UG,A((r~~f/Ecd(eD$%d Q"7eijlklem@n]a]^`a!U5QR8UVEWAX@n oo@ns#!D ct|}|dz t||z }|jddt|||dz tj |dz }|j r$tj|j}n[tt j|dz |j|j}|jtj|t!||||j"|||S)NtracksT)parentsexist_okr0zlid.json)rrhmkdirrr rpexistsjsonloads read_textrrr3r8r9 write_textdumpsrr ) rrrworkr:rdlid_filers r,_openrhsTA x%e,,AGGD4G($q:~vyy9:~Hjj++-.fkk!j.93;N;NPSP[P[\DJJsO, tUAGGENAs ;;r.c|jdr|jddtk\r|dSt|xs|jdS)z^Language code we believe a stream is in: confident detection > container tag > weak detection.rrjr)rN LID_MIN_PROBr ) stream_tagrs r,effective_langrvsA wwv37761-=6{ :  1#''&/1r.c tt|t|}t|tjz}||kr|}t d|dz|dzz }||||zj tj||||zj tj}}tj|dzj|dzjz} | dkryt||dddd} t|tjz|dz } } ttj| t d| | z | | zdzj | z tk\S)Nrg& .>Ffullr)minlenrr rpmaxastypersfloat64sqrtsumrrvabsPASSTHROUGH_CORR) arEwindowmax_lagnwmidxaxbdenomrkrs r,is_passthroughr}s+ CFCFA FVYY A1u  aa!q& !C s37^ " "2:: .#cAg0E0Ebjj0QB GGR1WMMOrQwmmo5 6E t|B4R4&)A w" #QUqA #aQ-A 23779EA BFV VVr.c|Dcgc]}|ds |ddd}}|Dcgc] }|dr | c}tdk(rnt|dk(r|dnd}t|dkDr>tfd|Ds*|jddd dd d |dg|fSdd|fg|fSg}D]td |r|jdd d d d1|r5tfd|Ds!|jdd d d |dh|jdt fd|Ddf||fScc}wcc}w)aPure selection rule. cands: [{"index", "lang", "passthrough"}]; wanted: requested base codes. Returns ([(index, declared base code or None)], [skipped {"index", "reason"}]). - passthrough copies of the original are never evaluated; - a lone remaining stream is always evaluated (even if it is in the wrong language: that is a defect); - otherwise streams in the source language are skipped and `wanted` filters by language. passthroughrzidentical to original audio)rreasonrrNc3BK|]}tdd|yw)rrNr).0rrests r, zchoose_dubs..s &YQ}T!WV_a'H&Ysz language rz not in zsame language as original ()c3<K|]}td|ywrNrrrrs r,rzchoose_dubs..sL ai ;Lsc3BK|]}td|s|ywrrrs r,rzchoose_dubs..s",^1-PQRXPY[\B]Q,^s)ranyrArnext)candssrcwantedrskippeddeclaredpickedrs ` @r, choose_dubsrsX]qRS`abo`p7/LMqGq 5!Am$4A 5D 4yA~ #F q 06!9d v;?3&YRX&Y#Y NNT!WW%59TRSWU[_L]]eflemAno pw; a!8,-w66+-F h 6C ( NNQwZ=XYZ[aYbXccd;ef g CLVLL NNQwZYqykQYZ`Ya;bc d MM1W:t,^,^`d'ef g h 7?!r 5s E  E  EEcZ|rt|dc|_|_|St|jj d|j }|&td|jd|jdt||_||j j dk(rdnd|_|S)Nrrzcannot determine language of z audio z; pass it explicitlyr tag) r rr!rrrNr ValueErrorrr)trr~s r,_assignrs ' 1:  H ahhll62AEE: <A^AEE&M155A73<!%%-QRQVQV3W%3CEE133%?A BB "%RUR_R_8`Q8`aOFG q c$i&7*<*@*@PQPWPWXYZaXbPcPgPghnPopqIO PAGF1Ivzz!}t < PRY YY5 E9a QsH 0H%0H* .,H/)rrrrrQr)g^@) r4rRrwrr9rrxrvrQr) rrrrrrrrrQr)r str | NonerrrQr)g4@r) rrRrErRrrvrrvrQrT)rz list[dict]rrrrSrQz/tuple[list[tuple[int, str | None]], list[dict]])rrrrrQr)rrrrrQr)rrrrrrrQztuple[list[Track], list[dict]])*__doc__ __future__rr`rloggingrd dataclassesrrpathlibrnumpyrs scipy.signalrr r rr r r rmediarrr getLoggerlogrrrrhrrrrrrrrrrZr.r,rs#  ($7755g "  1515 15hN p <2 W6  J$Zr.