o "che@sddlmZddlmZmZddlmZddlZddlm Z ddl m Z m Z m Z ddlmZmZmZmZmZdd lmZmZdd lmZd d lmZmZmZmZmZmZdd l m!Z!m"Z"ddl#m$Z$m%Z%ddl&m'Z'm(Z(ddl)m*Z*m+Z+ddl,m-Z-ddl.m/Z/ddl0m1Z1ddl2m3Z3ddl4m5Z5ddl6m7Z7ddgZ8Gddde!Z9Gddde"Z:GdddZ;GdddZdS)#) annotations)IterableOptional)LiteralN)_legacy_response)eval_list_paramseval_create_paramseval_update_params) NOT_GIVENBodyQueryHeadersNotGiven)maybe_transformasync_maybe_transform)cached_property)Runs AsyncRunsRunsWithRawResponseAsyncRunsWithRawResponseRunsWithStreamingResponseAsyncRunsWithStreamingResponse)SyncAPIResourceAsyncAPIResource)to_streamed_response_wrapper"async_to_streamed_response_wrapper)SyncCursorPageAsyncCursorPage)AsyncPaginatormake_request_options)EvalListResponse)EvalCreateResponse)EvalDeleteResponse)EvalUpdateResponse)EvalRetrieveResponse)MetadataEvals AsyncEvalsc @eZdZed7ddZed8ddZed9d d Zeed d d ed d:ddZd d d ed d;d$d%Z eed d d ed dd5d6Z d S)?r(returnrcC t|jSN)r_clientselfr1Z/home/air/goalskill/back/venv/lib/python3.10/site-packages/openai/resources/evals/evals.pyruns& z Evals.runsEvalsWithRawResponsecCt|Sa This property can be used as a prefix for any HTTP method call to return the raw response object instead of the parsed content. For more information, see https://www.github.com/openai/openai-python#accessing-raw-response-data-eg-headers )r5r/r1r1r2with_raw_response*zEvals.with_raw_responseEvalsWithStreamingResponsecCr6z An alternative to `.with_raw_response` that doesn't eagerly read the response body. For more information, see https://www.github.com/openai/openai-python#with_streaming_response )r:r/r1r1r2with_streaming_response4zEvals.with_streaming_responseNmetadataname extra_headers extra_query extra_bodytimeoutdata_source_config#eval_create_params.DataSourceConfigtesting_criteria-Iterable[eval_create_params.TestingCriterion]r?Optional[Metadata] | NotGivenr@str | NotGivenrAHeaders | NonerB Query | NonerC Body | NonerD'float | httpx.Timeout | None | NotGivenr#c Cs0|jdt||||dtjt||||dtdS) Create the structure of an evaluation that can be used to test a model's performance. An evaluation is a set of testing criteria and the config for a data source, which dictates the schema of the data used in the evaluation. After creating an evaluation, you can run it on different models and model parameters. We support several types of graders and datasources. For more information, see the [Evals guide](https://platform.openai.com/docs/guides/evals). Args: data_source_config: The configuration for the data source used for the evaluation runs. Dictates the schema of the data used in the evaluation. testing_criteria: A list of graders for all eval runs in this group. Graders can reference variables in the data source using double curly braces notation, like `{{item.variable_name}}`. To reference the model's output, use the `sample` namespace (ie, `{{sample.output_text}}`). metadata: Set of 16 key-value pairs that can be attached to an object. This can be useful for storing additional information about the object in a structured format, and querying for objects via API or the dashboard. Keys are strings with a maximum length of 64 characters. Values are strings with a maximum length of 512 characters. name: The name of the evaluation. extra_headers: Send extra headers extra_query: Add additional query parameters to the request extra_body: Add additional JSON properties to the request timeout: Override the client-level default timeout for this request, in seconds /evalsrErGr?r@rArBrCrDbodyoptionscast_to)_postrr EvalCreateParamsr!r# r0rErGr?r@rArBrCrDr1r1r2create=s0 z Evals.createrReval_idstrr&cC4|s td||jd|t||||dtdS)L Get an evaluation by ID. Args: extra_headers: Send extra headers extra_query: Add additional query parameters to the request extra_body: Add additional JSON properties to the request timeout: Override the client-level default timeout for this request, in seconds 6Expected a non-empty value for `eval_id` but received /evals/rRrUrV ValueError_getr!r&r0r[rArBrCrDr1r1r2retrieve~zEvals.retriever%c CsD|s td||jd|t||dtjt||||dtdS) Update certain properties of an evaluation. Args: metadata: Set of 16 key-value pairs that can be attached to an object. This can be useful for storing additional information about the object in a structured format, and querying for objects via API or the dashboard. Keys are strings with a maximum length of 64 characters. Values are strings with a maximum length of 512 characters. name: Rename the evaluation. extra_headers: Send extra headers extra_query: Add additional query parameters to the request extra_body: Add additional JSON properties to the request timeout: Override the client-level default timeout for this request, in seconds r_r`r?r@rRrS)rcrWrr EvalUpdateParamsr!r%r0r[r?r@rArBrCrDr1r1r2updates"z Evals.updateafterlimitorderorder_byrArBrCrDrnroint | NotGivenrp!Literal['asc', 'desc'] | NotGivenrq.Literal['created_at', 'updated_at'] | NotGiven SyncCursorPage[EvalListResponse]c C6|jdttt||||t||||dtjdtdSa List evaluations for a project. Args: after: Identifier for the last eval from the previous pagination request. limit: Number of evals to retrieve. order: Sort order for evals by timestamp. Use `asc` for ascending order or `desc` for descending order. order_by: Evals can be ordered by creation time or last updated time. Use `created_at` for creation time or `updated_at` for last updated time. extra_headers: Send extra headers extra_query: Add additional query parameters to the request extra_body: Add additional JSON properties to the request timeout: Override the client-level default timeout for this request, in seconds rP)rnrorprq)rArBrCrDquery)pagerUmodel) _get_api_listrr"r!rrEvalListParams r0rnrorprqrArBrCrDr1r1r2list&$z Evals.listr$cCr])I Delete an evaluation. Args: extra_headers: Send extra headers extra_query: Add additional query parameters to the request extra_body: Add additional JSON properties to the request timeout: Override the client-level default timeout for this request, in seconds r_r`rRrarc_deleter!r$rer1r1r2delete rgz Evals.delete)r+r)r+r5)r+r:rErFrGrHr?rIr@rJrArKrBrLrCrMrDrNr+r# r[r\rArKrBrLrCrMrDrNr+r&r[r\r?rIr@rJrArKrBrLrCrMrDrNr+r%)rnrJrorrrprsrqrtrArKrBrLrCrMrDrNr+ru r[r\rArKrBrLrCrMrDrNr+r$ __name__ __module__ __qualname__rr3r8r<r rZrfrlr~rr1r1r1r2r(%P    G%6?c @r*)?r)r+rcCr,r-)rr.r/r1r1r2r3.r4zAsyncEvals.runsAsyncEvalsWithRawResponsecCr6r7)rr/r1r1r2r82r9zAsyncEvals.with_raw_responseAsyncEvalsWithStreamingResponsecCr6r;)rr/r1r1r2r<<r=z"AsyncEvals.with_streaming_responseNr>rErFrGrHr?rIr@rJrArKrBrLrCrMrDrNr#c s>|jdt||||dtjIdHt||||dtdIdHS)rOrPrQNrRrS)rWrr rXr!r#rYr1r1r2rZEs 0  zAsyncEvals.createrRr[r\r&c<|s td||jd|t||||dtdIdHS)r^r_r`rRraNrbrer1r1r2rf zAsyncEvals.retriever%c sR|s td||jd|t||dtjIdHt||||dtdIdHS)rhr_r`riNrRrS)rcrWrr rjr!r%rkr1r1r2rls " zAsyncEvals.updatermrnrorrrprsrqrtCAsyncPaginator[EvalListResponse, AsyncCursorPage[EvalListResponse]]c Crvrw)r{rr"r!rrr|r}r1r1r2r~rzAsyncEvals.listr$cr)rr_r`rRraNrrer1r1r2rrzAsyncEvals.delete)r+r)r+r)r+rrrr)rnrJrorrrprsrqrtrArKrBrLrCrMrDrNr+rrrr1r1r1r2r)-rc@$eZdZd ddZed dd Zd S) r5evalsr(r+NonecCP||_t|j|_t|j|_t|j|_t|j|_t|j|_dSr-)_evalsrto_raw_response_wrapperrZrfrlr~rr0rr1r1r2__init__6  zEvalsWithRawResponse.__init__rcC t|jjSr-)rrr3r/r1r1r2r3I zEvalsWithRawResponse.runsNrr(r+r)r+rrrrrrr3r1r1r1r2r55 r5c@r) rrr)r+rcCrr-)rrasync_to_raw_response_wrapperrZrfrlr~rrr1r1r2rOrz"AsyncEvalsWithRawResponse.__init__rcCrr-)rrr3r/r1r1r2r3brzAsyncEvalsWithRawResponse.runsNrr)r+r)r+rrr1r1r1r2rNrrc@r) r:rr(r+rcCF||_t|j|_t|j|_t|j|_t|j|_t|j|_dSr-)rrrZrfrlr~rrr1r1r2rh  z#EvalsWithStreamingResponse.__init__rcCrr-)rrr3r/r1r1r2r3{rzEvalsWithStreamingResponse.runsNr)r+rrr1r1r1r2r:grr:c@r) rrr)r+rcCrr-)rrrZrfrlr~rrr1r1r2rrz(AsyncEvalsWithStreamingResponse.__init__rcCrr-)rrr3r/r1r1r2r3rz$AsyncEvalsWithStreamingResponse.runsNr)r+rrr1r1r1r2rrr)? __future__rtypingrrtyping_extensionsrhttpxrtypesrr r _typesr r r rr_utilsrr_compatr runs.runsrrrrrr _resourcerr _responserr paginationrr _base_clientr r!types.eval_list_responser"types.eval_create_responser#types.eval_delete_responser$types.eval_update_responser%types.eval_retrieve_responser&types.shared_params.metadatar'__all__r(r)r5rr:rr1r1r1r2s>