AjdZddlmZddlZddlZddlZddlmZddlZ ddl m Z ddl m Z mZmZej dZd Zdd Zddd Zddd Z d dd ZdddZy)aAudio layer: background-bed integrity and delivery-energy mismatch. Works on the raw mixes by default. If Demucs stems are available (`--separate`), vocals are used for energy comparison and the accompaniment stems for the background check, which removes the speech bias. ) annotationsN)Path) fftconvolve)SRloadmaskdubevalg{Gz?c$|dz |jz }||jdz ||jdz }}|jr|js|dz jsMtjtj dddddd d t |d t |t |g d d |dz |f|dz |ffD]F\}}tjdddddt |dddt tt |g d H||fS)zPhtdemucs two-stem split on CPU; returns (vocals, no_vocals) as 16 kHz mono wavs.htdemucsz .vocals.wavz.bed.wavz vocals.wavz-mdemucsz-nz --two-stemsvocalsz-jz-oT)checkcapture_outputz no_vocals.wavffmpegz-verrorz-yz-iz-ac1z-ar)r)stemexists subprocessrunsys executablestrr)wav44out_dirthreadsrvocals16bed16srcdsts 1/root/repos/pocs/video-evals/src/dubeval/audio.pyseparater#s Z %** ,D5::,k ::G T\F]<]eH OO %,,.|#++- NNCNND(D*m]e #g,c'lCJPW[lp r-x84/;QSX:YZ 'HC NNHdGT4S5RUW\^abd^egjkngop!% ' ' U?cRt|tztttz}}t||z}t j |j tjdzt j||z dd||z|}dt j|dzzS)z dBFS envelope at HOP resolution.samemodeN g|=) intrHOPlennpconvolveastypefloat64oneslog10)xwinnhframessqs r"rms_dbr:$s sRx=#cBh-qA Vq[F QXXbjj)Q. QV L_vXYz[\_ ]B e$ $$r$c||jz ||jz }}t||dddd}t|dz }t|tz }|||z ||zdz}t|j } || t j|dzj|dzjzdzz } | |z tzt| fS)zULag (s) of b relative to a and normalised correlation peak, on downsampled envelopes.Nfullrr&g& .>) meanrr-r+r,argmaxr.sqrtsumfloat) ab max_shifteaebrmidkr5ipeaks r" best_offsetrM,s \1qvvx<BB4R4&)A b'A+C IOA C!GS1Wq[ !C CJJLA q6RWWbAg]]_a}}>?$F GD ES=%+ %%r$c*t|t|} }tt|t| } |d| | d| } }t|dt| d| dt| d\} } |rt j | t } n*t||td| t||td| z} |dkD}| |z|| z |kDz}d}t j|jtt j ||z ddkD}g}t jt jdg|jtjdgg}tt j |d k(t j |d k(D]\}}||z tz|kr| ||j#r*tt j$|| z ||| ||n#tt j$|| z ||}|j'd d |d kDs||z tzdkDrdnd|rdndt)|tzdt)|tzdd|dd||z tzdddt)|d |dd || dkDrt)| dzd fSdfS)a Flag windows where the dub's non-speech bed is much quieter than the original's. Only frames where neither track has speech are compared, so speech loudness differences don't count (unless stems are used, in which case every frame is a fair comparison). Nipi2r'r(?rrr<BACKGROUND_DROPOUTaudiomajorminorgffffff?g?r&zBackground music/effects are ~z.0fz dB quieter in the dub for z.1fzs.zQRe-mix using the original background stem for this window (ducking or stem loss).)median_db_dropstems) categorysourceseverity confidencestartend descriptionfix_hintevidenceg333333?i)r:minr-rMr.r2boolr r,r/r0rBdiff concatenateint8zip flatnonzeroanymedianappendround)orig_beddub_bed orig_segsdub_segsduration using_stemsmin_lendrop_dbeoedr6offsetrLvalidaudibledroprJissuesdrCrDdeltas r"background_checkr~8sH vgB CGSWA VRVBr.C4L12nAt 3EFLFD4 y(C0!4tHhPS7TUWVW7XXY3hG 7?rBw'1 2D A ;;t{{5)2771:> G# MD F T[[%9A3?@AABNN16*BNN17,CD  1 ES=7 " @Ea @Pbiib!AuQqz :;.levelasU1s7|SS!121!Ah03CuR]]3+,DbffDr$g?g?rr&gư>ENERGY_MISMATCHrRrVrPz%Original line is delivered loudly (z=z+.1fz) but the dub is flat (z=z).zLRegenerate with a higher-intensity/emotional voice style (shouting/excited).) orig_level_z dub_level_z) rYrZr[r\r]r^ line_indexr_r`ra)r:dubcoveragedurr]r^ dub_startdub_endr-r.array nanmediannanstdrgrkrlindexrB)linesorig_vocdub_voczrurvrlnrowsrHor|zozdr{_rCrDs r"energy_mismatchr]s H vgBE  Prvv"++*<3r288RVV,eB bjj.Q R PD P 4y1}  %1!A$%&A %1!A$%&A bll1o "))A,"5 6B bll1o "))A,"5 6B Fb"- QAq q5AE>a1f MM-gehrxx+E"&&!4DTVT\T\!FqhNghijngooqrj-258Q-?PUV[\]V^`aPbc   M% P&%sA.G<1 H H)rrrrrr+returnztuple[Path, Path])g?)r4 np.ndarrayr5rBrr)g@)rCrrDrrErBrztuple[float, float])g?g(@)rmrrnrrqrBrrrcrsrBrtrBrztuple[list[dict], float | None])g?)rrrrrrBrz list[dict])__doc__ __future__rloggingrrpathlibrnumpyr. scipy.signalrspeechrrr getLoggerlogr,r#r:rMr~rr$r"rs # $""g "  % &PT"C"&"C16"CGL"CXw"CJr$