Ij/FdZddlmZddlZddlZddlZddlZddlZddlZddl Z ddl Z ddl Z ddl m Z mZddlmZej"dZej&dZej&dZdd Zdd Zdd Zdd Zdd ZddZ ddZddZddZddZdddZ y) adubeval-project: file-based multi-video workspace (no database). Layout (default root ./projects, override with --root or DUBEVAL_PROJECTS): projects/ index.html # all projects, latest score per pair, links to reports / project.json # sources, languages, run history media/original. # downloaded file, or symlink to a local file media/dub-01. ... work/ # per-track caches (audio, LID, ASR, stems, judge) report.json report.html # latest run runs//report.json # every run kept for comparison Sources may be local paths, http(s) URLs or Google Drive share links. ) annotationsN)datetimetimezone)Pathdubevalz^[a-z0-9][a-z0-9._-]{0,63}$z$drive\.google\.com|docs\.google\.comcbt|xs#tjjdxsdS)NDUBEVAL_PROJECTSprojects)rosenvironget)args 4/root/repos/pocs/video-evals/src/dubeval/projects.py_rootr%s% Grzz~~&89GZ HHcftjtjj dS)Nz%Y%m%dT%H%M%SZ)rnowrutcstrftimerr_nowr)s! << % . ./? @@rcttjj|jj j }tjd|xsdr|SdS)Nz\.[a-z0-9]{2,4}.mp4) rurllibrequesturlparsepathsuffixlowerre fullmatch)sourcers r_extr$-sM &..))&166 7 > > D D FF\\"4flC6OOrctjd|tjst|j j }|j std||j|jjxsd}|js|j||S|jt|}|jr|jjdkDr|S|j|jdz}t j#d||t$j'|r0ddl}|j+|t-|dd t/d |t0j2j5|d d i }t0j2j7|d5}t9|d5}t;j<||ddddddd|jjdk(r|j?t/d||jA||S#1swY_xYw#1swYcxYw)z`Materialize a source next to `dest_stem` (extension added). Idempotent: reuses an existing file.z ^https?://zno such file: rrz.partzdownloading %s -> %sNT)quietfuzzyz=Google Drive download failed (is the file shared publicly?): z User-Agentz dubeval/0.1)headers<)timeoutwbi)lengthzempty download: )!r!matchIr expanduserresolveis_fileFileNotFoundError with_suffixrr exists symlink_tor$statst_sizeloginfo_GDRIVEsearchgdowndownloadstr RuntimeErrorrrRequesturlopenopenshutil copyfileobjunlinkrename) r# dest_stemsrcdestpartr<reqrfs rfetchrN2s 88M6244 06l%%'//1{{}#nVH$=> >$$SZZ%5%5%7%A6B{{} OOC   f .D {{},,q0   DKK'1 2DHH #VT2~~f >>&#d)4t> D L!^_e^fgh hnn$$VlM5R$S ^^ # #C # 4 54d;K 5q   q!G 4 5 5 yy{a -fX677KK K  5 5 5 5s$ II(II IIc|dz }|jstd|dtj|j S)N project.jsonzno project at z' (create it with `dubeval-project add`))r4 SystemExitjsonloads read_text)drMs r_loadrVQs? NA 88:>!,STUU ::akkm $$rc|dz }|jtj|dd|j|dz y)Nzproject.json.tmpF)indent ensure_asciirP) write_textrRdumpsreplace)rUmetatmps r_saver`Xs7  CNN4::d15ABKKN"#rc ddlm}tj|st d|d|r|gng|zD] }|| ||z }|dz j } | r |s t d| r |s t d|dz } | j d d | r|td gd ggd n%tj|dz j} |rU| d r#| d d|k7rt d|d| d dd|tt|| d z j|d| d <| dD chc]} | d } } |D]i}|| vrt|| dt| ddzdz }| dj|t|j|d| j!|k |r|| d<|r|| d<t)|| |Scc} w#t"$r| rt%j&|d wxYw)z[Create a project or append dubs to an existing one; downloads happen here, not at run time.)r0zinvalid project name z*: lowercase letters, digits, '.', '_', '-'rPza new project needs --originalz&a new project needs at least one --dubmediaTparentsexist_okN)namecreatedoriginaldubssrc_lang tgt_langsrunsrir#zproject z already has an original (z); use a new project name)r#filerjzdub-02d) ignore_errorsrkrl)langr0_NAMEr-rQr4mkdirrrRrSrTr>rN relative_tolenappendadd BaseExceptionrCrmtreer`)rootrgrirjrkrlr0labelrUis_newrcr^xknownrHrMs rrwrw^sC ;;t 08bcdd (8*bI= t An$,,. .F h9:: dABB KE KKtK,!TVrW[jl '+zz1~3E2P2P2R'S  JD$4X$>($J 8D61KDQ[L\]eLfKgh:":;;*2ChPUXbPb@c@o@opq@rrtreportsrtargetsummaryrrruskipped_tracksrr` write_index)rzrg cfg_overridesr run_batchrrUr^cfgr}batchstamprun_dirpayloadrLs rrrs2 t A 8D  g$((:.$((;:O:USU gYf gC a$z*622DQWL4YqQ6]4Y[\_e[egj kE FE&j5 G MM$M.##1#-G }((1""7+ 5    (!m*;<LsGm$;#H#H#KLdidqdqs_`66!((6*:188TaKbIIOO @Y@Y[su++, FG  !TN < 5Zss G .AG c g}t|jdD]5} |jtj|j 7|S#tj $rtjd|YewxYw)Nz*/project.jsonz unreadable %s) sortedglobrvrRrSrTJSONDecodeErrorr8warning)rzoutrMs rr r sq C DII./ 0, , JJtzz!++-0 1, J## , KK + ,s2A)BBcg}t|D] }|jdr|ddnd}djd|xsijdgDxsd}djd|xsijd gD}tj|d }|r d |d |d nd}|j d|dtj|dddt |dd||d|rtj|dnddt |jdgd|d#ddj|xsdd}|jdd|dz j|d |dz S)!Nrmrz
c 3K|]F}tj|dd|dd|dd|ddd|dd d |dd d Hy w)r a:ru — rz (Crcriticalz / Mmajorz / mminor)Nhtmlescape).0r}s r zwrite_index..s4{{1V9%&c!M*:);8AgJ<P$%j12$q9J7KG7T6UUYZ[\mZnovZwYxxy {4sA Arz not run yetrc3K|]9}dtj|ddtj|dd;yw)z
failed: r: errorz
Nr)rrMs rrzwrite_index..sIB4DKK%4I3J"T[[YZ[bYcMdLeeklBs?Arrgz report · jsonz z
rir#z
rjz rz
z run(s)
z a.dubeval projects

dubeval projects

z'z
projectdubslatest scores per pair last runopen
no projects
Trd index.htmlzutf-8)encoding) r r joinrrrvrursr[) rzrowsplastrrnlinkpages rrrs D d^ - !f qy}4 4jb%%gr244E8E B#':2"2"28R"@BB KK& "`d1#B1#E[\jl k!$5dkk!J-PXBY6Z5[\qy>*)E76(C6:4;;tDz2C=QTUVUZUZ[aceUfQgPhi& , - -3 46774=3mDm2no  "D JJtdJ+ L$$TG$< , rc tjdttj}|j dd|j dd}|j d d }|j d |j d d |j ddgd|j d|j dd|j ddd|j dd}|j d dd||fD]i}|j ddd|j d d!|j d"d#d$%|j d&d'd(%|j d)td*+k|j d,d-|j d.d/|j|tjtjd0tj1d2D]4}tj|jtj 6t#j$}fd3}j&d k(rj(xsd4j+d5D cgc]#} | j-s| j-%} } t/|j0j2j4j6| } t?d6 j@r j0gng} nj&dk(r9j0d7gk(rtC|D cgc]} | d  c} n j0} nj&d,k(rtC|D]{} | jEd8r| d8d9nd}|rd:jGd;|d<Dnd=}t?| d d>d?tI| d@dAtI| jEd8gdB|}yCt?dDtK|yCdC}| D]}tA|||}|d<D]'}t?|dE|dFdG|dHdI|dJdB|dK )|dLD]-}t?|dM|dNdE|dOtjPdQ}/t?|dR||z dSz | rt?dT|dUz |Scc} w#t8$r%} |j;t=| Yd} ~ d} ~ wwxYwcc} w)VNzdubeval-project)prog descriptionformatter_classz--rootz.overridessnn" MM5GTXT`T`$'IIK6$9bSVS\S\]`Sa?oaefelelen ?o+/>>+?+?+AV+K4a aQUQ_Q_a a?os B5 2B5 rrzproject ready: allrmrz, c36K|]}|dd|dyw)r=rNr)rys rrzmain..s#Rq!F)Aaj\:Rsrznot run24z dubs=rjz runs= rzwrote rrrrz score rrrz : FAILED rr)rnrbz : report rzindex: r)&argparseArgumentParser__doc__RawDescriptionHelpFormatter add_argumentadd_subparsers add_parserint parse_argslogging basicConfigINFOsysstderr getLoggersetLevelWARNINGrrzrtgt_langrrrwrgrirrk ValueErrorrr>printrr r rrur)argvrsubarLspnoisyrzrr}tgtrUenamesrscoresstatusrresrrMrs @rmainr s%6G080T0T VANN8"`Na    5C u#TUANN6NN<&NNONN78R>cNdNN< NN<&JNKNN7<6WNX u#YZANN6+FNG!f:  \@lm  l;  /LSde  /FMbc  #q9 :NN6 ANBNN7!>N? << D gll3LUXU_U_`^;% ))'//:;  Da  xx5#'==#6B"="=c"BPQaggiqwwyPP D$))T]]DHHdmmSQA s#$#xx R U 7;yyUG7KHTN3q63QUQZQZ V $ bA$%EE&M1V9R=tDVZTYYRDMRR`iF QvYrN&QvY(8s155QSCT?U>VVWX^W_` a b {4()*+ F 9$9;'W fA QCr!F)C-(8'97 AaPaNbMcd e fX A QCy5 "QwZL9 KF  9TAX 56789  |+,-. M=Q  GGCFOO  4s*R 1R 8R S R>R99R>)r str | Nonereturnr)rr>)r#r>rr>)r#r>rGrrr)rUrrdict)rUrr^rrNone)rzrrgr>rirrj list[str]rkrrlrrr)rzrrgr>rrrr)rzrrz list[dict])rzrrr)N)rzlist[str] | Nonerr)!r __future__rrrrRrr r!rCrurllib.requestrrrpathlibrrr8compilerrr:rrr$rNrVr`rwrr rr rrrrs #  'g " 12 "**< =IAP >%$ ) ) !%) X28Er