/je UdZddlZddlZddlZddlmZmZmZmZm Z m Z m Z ddl Z ddlmZddlmZddlmZddlmZddlmZdd lmZdd lmZdd lmZdd lmZdd lmZddlm Z ddlm!Z!ddlm"Z"ddlm#Z#ddlm$Z$ddl%m&Z&ddl%m'Z'ddl(m)Z)ddl(m*Z*ddl+m,Z,ddl+m-Z-ddl+m.Z.ddl+m/Z/ddl+m0Z0dZ1ejdejfejhejjejlejne'jpdZ9de:de ee:fd Z;ejxGd!d"ejzZ>id#e>j~d$e>jd%e>jd&e>jd'e>jd(e>jd)e>jd*e>jd+e>jd,e>jd-e>jd.e>jd/e>jd0e>jd1e>jd2e>jd3e>jid4e>jd5e>jd6e>jd7e>jd8e>jd9e>jd:e>jd;e>jdjd=e>jd>e>jd?e>jd@e>jdAe>jdBe>jdCe>jdDe>je>je>je>je>je>je>je>je>je>jdE ZLee:d"feMdF<GdGdHZNy)IaMediaPipe SolutionBase module. MediaPipe SolutionBase is the common base class for the high-level MediaPipe Solution APIs such as BlazeFace, hand tracking, and BlazePose. The SolutionBase class contains the shared logic among the high-level Solution APIs including graph initialization, processing image/audio data, and graph shutdown. Thus, users can easily create new MediaPipe Solution APIs on top of the SolutionBase class. N)AnyIterableListMapping NamedTupleOptionalUnion) containers) descriptor)message)#constant_side_packet_calculator_pb2)#image_transformation_calculator_pb2)$tensors_to_detections_calculator_pb2)"landmarks_smoothing_calculator_pb2)logic_calculator_pb2)thresholding_calculator_pb2)calculator_pb2) body_rig_pb2)classification_pb2) detection_pb2) landmark_pb2)rect_pb2)annotation_data_pb2)-lift_2d_frame_annotation_to_3d_calculator_pb2)packet_creator) packet_getter)calculator_graph) image_frame)packet) resource_util)validated_graph_config)ConstantSidePacketCalculatorImageTransformationCalculatorLandmarksSmoothingCalculatorLogicCalculatorThresholdingCalculatorTensorsToDetectionsCalculator#Lift2DFrameAnnotationTo3DCalculatoroneof_type_namereturnc|jdrV|jdrE|tdtd }|jdDcgc]}|j c}Sycc}w)NzOneOf<>,) startswithendswithlensplitstrip)r*comma_separated_typesns a/root/repos/pocs/video-evals/.venv/lib/python3.12/site-packages/mediapipe/python/solution_base.pytype_names_from_oneofr7Rs_)o.F.Fs.K+CM3s8)D4::3? @!AGGI @@  AsA)c^eZdZdZdZdZdZdZdZdZ dZ d Z d Z d Z d Zd ZdZededdfdZy)PacketDataTypez:The packet data types supported by the SolutionBase class.stringbool bool_listintint_listfloat float_listmatriximage image_listrproto proto_listregistered_namer+c t|S#t$rA}t|}|r-|D](}|tjvst|ccYd}~S|d}~wwxYwN) NAME_TO_TYPEKeyErrorr7keys)rFenamesr5s r6from_registered_namez#PacketDataType.from_registered_nameks` / ** #O4e  #A ,##% %? " # g s% A(A AA AAN)__name__ __module__ __qualname____doc__STRINGBOOL BOOL_LISTINTINT_LISTFLOAT FLOAT_LISTAUDIOIMAGE IMAGE_LIST IMAGE_FRAMEPROTO PROTO_LIST staticmethodstrrNr6r9r9ZsgB & $) # ( %* % %*+ %* C 4D  rcr9r:r;z::std::vectorr=z::std::vectorint64int64_tz::std::vectorz::std::vectorr?z::std::vectorz::mediapipe::Matrixz::mediapipe::ImageFramez::mediapipe::Classificationz::mediapipe::ClassificationListz)::mediapipe::ClassificationListCollectionz::mediapipe::Detectionz::mediapipe::DetectionListz::mediapipe::Landmarkz::mediapipe::LandmarkListz#::mediapipe::LandmarkListCollectionz::mediapipe::NormalizedLandmarkz::mediapipe::FrameAnnotationz::mediapipe::Triggerz::mediapipe::Rectz::mediapipe::NormalizedRectz#::mediapipe::NormalizedLandmarkListz-::mediapipe::NormalizedLandmarkListCollectionz::mediapipe::Imagez!::std::vector<::mediapipe::Image>z*::std::vector<::mediapipe::Classification>z.::std::vector<::mediapipe::ClassificationList>z%::std::vector<::mediapipe::Detection>z)::std::vector<::mediapipe::DetectionList>) z$::std::vector<::mediapipe::Landmark>z(::std::vector<::mediapipe::LandmarkList>z.::std::vector<::mediapipe::NormalizedLandmark>z2::std::vector<::mediapipe::NormalizedLandmarkList>z ::std::vector<::mediapipe::Rect>z*::std::vector<::mediapipe::NormalizedRect>z::mediapipe::Jointz::mediapipe::JointListz%::std::vector<::mediapipe::JointList>rIceZdZdZ d!deedeejdeeee fdee jdeeee fdee ed eeee ffd Zd eej"eeeej"e jfffd efd Zd"dZd"dZ d#dej.deeee fdee ed eeee ffdZdejdeee fd dfdZde jdeee fd e jfdZdej8de jd dfdZde de d ej>fdZ de dej>d e fdZ!dZ"d Z#y)$ SolutionBaseaThe common base class for the high-level MediaPipe Solution APIs. The SolutionBase class contains the shared logic among the high-level solution APIs including graph initialization, processing image/audio data, and graph shutdown. Example usage: with solution_base.SolutionBase( binary_graph_path='mediapipe/modules/hand_landmark/hand_landmark_tracking_cpu.binarypb', side_inputs={'num_hands': 2}) as hand_tracker: # Read an image and convert the BGR image to RGB. input_image = cv2.cvtColor(cv2.imread('/tmp/hand.png'), COLOR_BGR2RGB) results = hand_tracker.process(input_image) print(results.palm_detections) print(results.multi_hand_landmarks) Nbinary_graph_path graph_configcalculator_params graph_options side_inputsoutputsstream_type_hintsc bt|t|k(r tdtjj tj j tjtjdd}tj|tj} |r1| jtj j ||n| j|j| |||} |rj| ||rj!| j"|t%j&| _d_i_dt.dt0j2d dffd } j4j7D]} j(j9| | d !|xsij;D cic]%\} }| j=j>| |'c}} _ j(jCj@ycc}} w) a Initializes the SolutionBase object. Args: binary_graph_path: The path to a binary mediapipe graph file (.binarypb). graph_config: A CalculatorGraphConfig proto message or its text proto format. calculator_params: A mapping from the {calculator_name}.{options_field_name} str to the field value. graph_options: The graph options protobuf for the mediapipe graph. side_inputs: A mapping from the side packet name to the packet raw data. outputs: A list of the graph output stream names to observe. If the list is empty, all the output streams listed in the graph config will be automatically observed by default. stream_type_hints: A mapping from the stream name to its packet type hint. Raises: FileNotFoundError: If the binary graph file can't be found. RuntimeError: If the underlying calculator graph can't be successfully initialized or started. ValueError: If any of the following: a) If not exactly one of 'binary_graph_path' or 'graph_config' arguments is provided. b) If the graph validation process contains error. c) If the registered type name of the streams and side packets can't be found. d) If the calculator options of the calculator listed in calculator_params is not allowed to be modified. e) If the calculator options field is a repeated field but the field value to be set is not iterable. f) If not all calculator params are valid. zBMust provide exactly one of 'binary_graph_path' or 'graph_config'.N)rh)rir stream_name output_packetr+c$|j|<yrH)_graph_outputs)rqrrselfs r6callbackz'SolutionBase.__init__..callbacks)6d+&rcT)"r; ValueErrorossepjoinpathabspath__file__r2r set_resource_dirr!ValidatedGraphConfig initialize_initialize_graph_interface_modify_calculator_options_set_extensionrkrCalculatorGraph_graph_simulated_timestamprtrarPacket_output_stream_type_inforKobserve_output_streamitems _make_packet_side_input_type_info_input_side_packets start_run)rurhrirjrkrlrmrn root_pathvalidated_graphcanonical_graph_config_protorvrqnamedatas` r6__init__zSolutionBase.__init__sP $|"44  N PP BGGOOH5;;BFFCCRHII""9-,AACO  GGLL4EF!H  l ;#'#C#Cg/@$B  %%&B&79 6DD')#2213DK !DD7c7&--7D74499;E  kk'' XtDE ',"335  D$ d : :4 @$GG D KK$223  s*H+ input_datar+c |jjt|tjri|j t dt|j jdk7r t dtt|j |i}n|}|xjdz c_ |j t d|jD]5\}}|j |}|tjk(s|tj k(r&t#d|j |j$d|tj&k(s|tj(k(rh|j*d t,k7r t d |jj/||j1||j3|j |jj/||j1||j3|j 8|jj5|j6 t d t9j:d |j6j}|j6jD]U}||jvr8t=|||j?|j6||j|It=||dW|S)a+Processes a set of RGB image data and output SolutionOutputs. Args: input_data: Either a single numpy ndarray object representing the solo image input of a graph or a mapping from the stream name to the image or proto data that represents every input streams of a graph. Raises: NotImplementedError: If input_data contains audio data or a list of proto objects. RuntimeError: If the underlying graph occurs any error. ValueError: If the input image data is not three channel RGB. Returns: A NamedTuple object that contains the output data of a graph run. The field names in the NamedTuple object are mapping to the graph output stream names. Examples: solution = solution_base.SolutionBase(graph_config=hand_landmark_graph) results = solution.process(cv2.imread('/tmp/hand0.png')[:, :, ::-1]) print(results.detection) results = solution.process( {'video_in' : cv2.imread('/tmp/hand1.png')[:, :, ::-1]}) print(results.hand_landmarks) Nz/_input_stream_type_info is None in SolutionBasezQCan't process single image input since the graph has more than one input streams.i5z_graph is None in SolutionBasezASolutionBase can only process non-audio and non-proto-list data. z type is not supported yet.z0Input image must contain three channel rgb data.)streamrz0_output_stream_type_info is None in SolutionBaseSolutionOutputs) rtclear isinstancenpndarray_input_stream_type_inforwr1rKnextiterrrrr9r_rZNotImplementedErrorrr]r[shape RGB_CHANNELSadd_packet_to_input_streamratwait_until_idler collections namedtuplesetattr_get_packet_content)rur input_dictrqrinput_stream_typesolution_outputss r6processzSolutionBase.process s< *bjj) % % -JKK T ) ) . . 0 1Q 6   d::;?) *+ + !;!; ; !5!5 5 ::a=L (MN N ..$$%6%)++-2d.G.G+H / J ..$$%6%)++-2d.G.G+H / J%J. KK! $$, I JJ"--488==?A4499;5 ++ + k  $ $T%B%B;%O%)%8%8%E G H  +t45 rcc|j td|jjd|_d|_d|_y)+Closes all the input sources and the graph.Nz1Closing SolutionBase._graph which is already None)rrwcloserrrus r6rzSolutionBase.closefs? {{ J KKKKDK#'D $(D!rcc|jr@|jj|jj|jyy)z!Resets the graph for another run.N)rrrrrs r6resetzSolutionBase.resetos6 {{ kk kkD445rcrc tj}|jjd fd}|jDcic]} |||c}|_|s |j }n|}|Dcic]} |||c}|_ fd} |xsijD cic]\}}  || |c} }|_ |Scc}wcc}wcc} }w)zSGets graph interface type information and returns the canonical graph config proto.c*|jddS)N:)r2)tag_index_names r6get_namez:SolutionBase._initialize_graph_interface..get_names  ! !# &r **rcc|}r|jvr|Stjj|SrH)rKr9rNregistered_stream_type_name)packet_tag_index_namerqrrnrs r6get_stream_packet_typezHSolutionBase._initialize_graph_interface..get_stream_packet_typesP23k {.?.D.D.FF --  0 0  5 5k BDDrccXtjj|SrH)r9rN registered_side_packet_type_name)rrrs r6get_side_packet_typezFSolutionBase._initialize_graph_interface..get_side_packet_types-  0 0  : :,- /00rc) rCalculatorGraphConfigParseFromString binary_config input_streamr output_streamrrr) rurrlrmrnrrroutput_streamsr_rs ` ` @r6rz(SolutionBase._initialize_graph_interfaceus$2#G#G#I  001N1NO+D;GG$   "8"HH$D 3AAnn-%   "8"HH%D!0#."3!:!:!<" NA  "6~"FF"D ('7$%"s C>C"9C'calculator_graph_configcd}d}||}d}|jD]}|j|vrtj|j}|t d|jd||j} |j dr%|jrt d|jd |jrd } |jD]v} | jjd d } | |jjk(s;|j| j} || | | j| _ d } n| sd|} || | |jjj!| n*||j"j$|j&| |dz }|t)|k(sn|t)|kr t dy)zNModifies the CalculatorOptions of the calculators listed in calculator_params.ci}|jD]`\}}|jd}t|dk7rtd|d|d}|d}||vr||j ||fY||fg||<b|S)N.rz The key "z&" in the calculator_params is invalid.rr)rr2r1rwappend)flat_map nested_map compound_name field_valuecalculator_and_field_namecalculator_name field_names r6!generate_nested_calculator_paramszRSolutionBase._modify_calculator_options..generate_nested_calculator_paramssj(0(8 D $-$1$7$7$<! ( )Q .-(NOQ Q3A6.q1 j ( _ % , ,j+-F G*4k)B(C*_ % Drcc|D]\}}||j||jj|j}|tj j k(r^t|tst|dt|d|j||D]}t||j|t|||y)Nz6 is a repeated proto field but the value to be set is z, which is not iterable.) ClearField DESCRIPTORfields_by_namelabelr FieldDescriptorLABEL_REPEATEDrrrwtypegetattrrr)calculator_optionsoptions_field_listrr field_labelelems r6modify_options_fieldszFSolutionBase._modify_calculator_options..modify_options_fieldss%7A !*k    ' ' 3*55DD%  J66EE Ek84L!""&{"3!44LNOO  ) )* 5# C(*5< y !d&7&76tyykB8 9: :    %%% Dmm))#.r2) ,11;; ;!-!8!8!D  !"46H I+==?DJ$( !  %+~   24F G      ! & &'9 : dll55l6F6FG0 2al 56 6 Q(Rc233 = >>4rcoptions_messagevaluesct|dr|j}|D]}|\}}|jd}|}t|dkDr!t ||d}|d=t|dkDr!t ||d}t|dr|dd=|j ||t|dr|j |t||d||S)zSets protobuf field values. Args: options_message: the options protobuf message. values: field value pairs, where each field may be a "." separated path. Returns: the options protobuf message. rrrrrNCopyFrom)hasattrrr2r1rextendrr) rurrpairfieldrfieldsmvs r6create_graph_optionsz!SolutionBase.create_graph_optionssvw||~f%nue{{3f a K!O Avay ! 1I K!O !VAY a H  aD  1j ! 56!9e$% rcextension_listextension_valuec|D]b}|j|jst|}|j||j ||j |y|j j |y)z:Sets one value in a repeated protobuf.Any extension field.N)IsrrUnpack MergeFromrr)rurr extension_anyrs r6rzSolutionBase._set_extension'sz(   /44 5 !D ! #Q O$1 o.rcpacket_data_typerc|tjk(s|tjk(r=ttd|j z|t jjSttd|j z|S)Ncreate_) image_format) r9r]r[rrrr ImageFormatSRGB)rurrs r6rzSolutionBase._make_packet4stN666N000 HW^Y1A1G1G%G H [4499;;IW^Y1A1G1G%G H NNrcrrch|jry|tjk(rtj|S|tj k(s|tj k(r1ttd|jz|jSttd|jz|S)zGets packet content from a packet by type. Args: packet_data_type: The supported packet data type. output_packet: The packet to get content from. Returns: Packet content by packet data type. None to indicate "no output". Nget_) is_emptyr9rSrget_strr]r[rr numpy_view)rurrrs r6rz SolutionBase._get_packet_content=s >000  " "= 11 n88 8 n22 2-W]F%++%,--:<#7#7.0&~';';.0 01E1E!.0"n22#.0$!."6"6%.0&^11'.0( !5!5).0**>+?+?+.0,&~';';-.0.#N$8$8/.00N001.02--3.04">#7#75.06*>+?+?7.084^5I5I9.0:...;.0<()B)B=.0>1.2K2K?.0@5n6O6OA.0B,^-F-FC.0D01J1JE.0F-;,E,E0>0I0I6D6O6O!!(6(A(A2@2K2K(..,22-;-F-F[.0 gc++,.bttrc