=encoding utf8

=head1 NAME

OpenAPI::Client::OpenAI::Path::evals - Documentation for the /evals path.

=head1 OPERATIONS

=head2 GET /evals

=head3 listEvals

  $client->list_evals({
      body => { ... },
  });

List evaluations for a project.

=head4 Path/query parameters

=over

=item * C<after> (in query, optional, string) - Identifier for the last eval from the previous pagination request.

=item * C<limit> (in query, optional, integer) - Number of evals to retrieve.

Default: 20

=item * C<order> (in query, optional, string) - Sort order for evals by timestamp. Use C<asc> for ascending order or C<desc>
for descending order.

Allowed values: asc, desc

Default: asc

=item * C<order_by> (in query, optional, string) - Evals can be ordered by creation time or last updated time. Use C<created_at>
for creation time or C<updated_at> for last updated time.

Allowed values: created_at, updated_at

Default: created_at

=back

=head4 Responses

B<200 - A list of evals>

Content-Type: application/json

Example:

    "{\n  \"object\": \"list\",\n  \"data\": [\n    {\n      \"object\": \"eval\",\n      \"id\": \"eval_67abd54d9b0081909a86353f6fb9317a\",\n      \"data_source_config\": {\n        \"type\": \"custom\",\n        \"schema\": {\n          \"type\": \"object\",\n          \"properties\": {\n            \"item\": {\n              \"type\": \"object\",\n              \"properties\": {\n                \"input\": {\n                  \"type\": \"string\"\n                },\n                \"ground_truth\": {\n                  \"type\": \"string\"\n                }\n              },\n              \"required\": [\n                \"input\",\n                \"ground_truth\"\n              ]\n            }\n          },\n          \"required\": [\n            \"item\"\n          ]\n        }\n      },\n      \"testing_criteria\": [\n        {\n          \"name\": \"String check\",\n          \"id\": \"String check-2eaf2d8d-d649-4335-8148-9535a7ca73c2\",\n          \"type\": \"string_check\",\n          \"input\": \"{{item.input}}\",\n          \"reference\": \"{{item.ground_truth}}\",\n          \"operation\": \"eq\"\n        }\n      ],\n      \"name\": \"External Data Eval\",\n      \"created_at\": 1739314509,\n      \"metadata\": {},\n    }\n  ],\n  \"first_id\": \"eval_67abd54d9b0081909a86353f6fb9317a\",\n  \"last_id\": \"eval_67abd54d9b0081909a86353f6fb9317a\",\n  \"has_more\": true\n}\n"

=head2 POST /evals

=head3 createEval

  $client->create_eval({
      body => { ... },
  });

Create the structure of an evaluation that can be used to test a model's
performance. An evaluation is a set of testing criteria and the config for a
data source, which dictates the schema of the data used in the evaluation.
After creating an evaluation, you can run it on different models and model
parameters. We support several types of graders and datasources. For more
information, see the
L<Evals guide|https://platform.openai.com/docs/guides/evals> .

=head4 Responses

B<201 - OK>

Content-Type: application/json

Example:

    "{\n  \"object\": \"eval\",\n  \"id\": \"eval_67abd54d9b0081909a86353f6fb9317a\",\n  \"data_source_config\": {\n    \"type\": \"custom\",\n    \"item_schema\": {\n      \"type\": \"object\",\n      \"properties\": {\n        \"label\": {\"type\": \"string\"},\n      },\n      \"required\": [\"label\"]\n    },\n    \"include_sample_schema\": true\n  },\n  \"testing_criteria\": [\n    {\n      \"name\": \"My string check grader\",\n      \"type\": \"string_check\",\n      \"input\": \"{{sample.output_text}}\",\n      \"reference\": \"{{item.label}}\",\n      \"operation\": \"eq\",\n    }\n  ],\n  \"name\": \"External Data Eval\",\n  \"created_at\": 1739314509,\n  \"metadata\": {\n    \"test\": \"synthetics\",\n  }\n}\n"

=head1 SCHEMAS

=head2 CreateEvalRequest

Properties:

=over

=item * C<data_source_config> (object, required) - The configuration for the data source used for the evaluation runs. Dictates
the schema of the data used in the evaluation.

=item * C<metadata> (Metadata)

See L</Metadata> below for shape.

=item * C<name> (string) - The name of the evaluation.

=item * C<testing_criteria> (array of object, required) - A list of graders for all eval runs in this group. Graders can reference
variables in the data source using double curly braces notation, like
C<{{item.variable_name}}> . To reference the model's output, use the C<sample>
namespace (ie, C<{{sample.output_text}}> ).

=back

=head2 Eval

Properties:

=over

=item * C<created_at> (integer, required) - The Unix timestamp (in seconds) for when the eval was created.

=item * C<data_source_config> (object, required) - Configuration of data sources used in runs of the evaluation.

=item * C<id> (string, required) - Unique identifier for the evaluation.

=item * C<metadata> (Metadata, required)

See L</Metadata> below for shape.

=item * C<name> (string, required) - The name of the evaluation.

=item * C<object> (string, required) - The object type.

Allowed values: eval

Default: eval

=item * C<testing_criteria> (array of object, required) - A list of testing criteria.

Default: eval

=back

=head2 EvalList

Properties:

=over

=item * C<data> (array of Eval, required) - An array of eval objects.

=item * C<first_id> (string, required) - The identifier of the first eval in the data array.

=item * C<has_more> (boolean, required) - Indicates whether there are more evals available.

=item * C<last_id> (string, required) - The identifier of the last eval in the data array.

=item * C<object> (string, required) - The type of this object. It is always set to "list".

Allowed values: list

Default: list

=back

=head2 Metadata

Set of 16 key-value pairs that can be attached to an object. This can be
useful for storing additional information about the object in a structured
format, and querying for objects via API or the dashboard.

Keys are strings with a maximum length of 64 characters. Values are strings
with a maximum length of 512 characters.

=head1 SEE ALSO

L<OpenAPI::Client::OpenAI::Path>

=head1 COPYRIGHT AND LICENSE

Copyright (C) 2023-2026 by Nelson Ferraz

This library is free software; you can redistribute it and/or modify
it under the same terms as Perl itself, either Perl version 5.14.0 or,
at your option, any later version of Perl 5 you may have available.

=cut
