📄 critic.cpython-312.pyc
/home/palash/git/local-ai/server/features/__pycache__/critic.cpython-312.pyc
Language: pyc • Lines: 671
�

�Y�j�M��<�dZddlZddlZddlZddlZddlmZdZejd�Z	ejd�Z
ejd�Zejdej�Z
d	Zdd
�Zd�Zd�Zd
�Zd�Zd�Zdd�Zd�Zd�Zd�Zd�Zd�Zdd�Zd�Zy)aiPost-generation self-verification ("critic") pass for research answers.

After the LLM emits its final research answer, ``run_verification`` extracts the
structured ``(Author, Venue, Year) [url]`` inline citations, re-fetches each
source and asks a fresh LLM call ("the critic") to check that (1) the source
exists, (2) the claimed author/venue/year metadata matches the real page, and
(3) the specific claim is actually supported by the source text. The resulting
verdict drives a small, local edit of just that sentence (never a full report
rewrite) and a transparent verification trail is appended to the answer.

The pass is always-on for research tasks (no separate toggle) and bounded only
per-citation: at most ``VERIFY_RETRIES`` extra search/fetch attempts per
citation, so a pathological report can never loop forever while keeping the
"no overall cap" behaviour.
�N)�Ma�You are a strict citation fact-checker for a research assistant. Given a claimed citation (URL plus author/venue/year metadata plus one specific claim) and the fetched text of the cited web page, determine: (1) whether the page exists and its topic matches the citation, (2) whether the cited author, venue and year actually appear on the page, (3) whether the exact claim is supported by the page text. Reply with ONLY a JSON object of the form: {"exists": bool, "url_ok": bool, "author_ok": bool, "venue_ok": bool, "year_ok": bool, "claim_support": "supports"|"partial"|"unsupported"|"absent", "corrected_meta": string|null, "reason": string}. "corrected_meta" must be a comma-separated "Author, Venue, Year" string when the metadata is wrong AND the page itself reveals the correct values, otherwise null. "year_ok"/"author_ok"/"venue_ok" must be false only when the page text contradicts the claim, not when the value is merely absent.zK(?P<meta>[\[(][^)\]\[(\n]{0,180}[\]\)])\s*\[(?P<url>https?://[^\s\]<>']+)\]z1(?P<meta>[\[(][^)\]\[(\n]{0,180}[\]\)])\s*\[\s*\]z(?<!\w)(https?://[^\s\]<>()]+)z	```.*?```z2

<details>
<summary>Source verification</summary>c	�
�tj|�d|d�d|d�g|ddd�}d}td�D]�}	tj|�t	j
tj|�|d	�
�}|j�|j�ddd
jd�}|r|cSd}td|�d|dz�d|���tjd���y#t$r}	t|	�}Yd}	~	�Hd}	~	wwxYw)u�Secondary, non-streamed, low-temperature LLM call. Retries once and
    never raises — returns None only when the model itself is unreachable.�system)�role�content�userg�������?F)�model�messages�
max_tokens�temperature�streamN��x)�json�timeout�choicesr�messagerzempty content in responsez[critic] LLM call failed (mode=z
, attempt �z/2): g�?)r�server_model_id�range�mark_slot_kv_dirty�requests�post�
server_url�raise_for_statusr�get�	Exception�str�print�time�sleep)
rr�moder�payload�last_err�attempt�rr�es
          �server/features/critic.py�_critic_completionr)7s���"�"�4�(��&�1���-�
�!���	�G��H���8���		�
� � ��&��
�
�a�l�l�4�0�w��L�A�
��� ��f�f�h�y�)�!�,�Y�7�;�;�I�F�G����2�H�	�/��v�Z��!��}�E�RZ�Q[�\�]��
�
�3�����	�	��1�v�H��	�s�A;C$�3C$�$	D�-C=�=Dc��|sytjdd|�j�}|jd�}|j	d�}|dks||kry	tj|||dz�S#ttf$rYywxYw)zFBest-effort JSON parse of the critic's reply. Returns None on failure.Nz```(?:json)?��{�}rr)	�re�sub�strip�find�rfindr�loads�
ValueError�	TypeError)�text�cleaned�start�ends    r(�_parse_verdictr:Us������f�f�_�b�$�/�5�5�7�G��L�L���E�
�-�-��
�C��q�y�C�5�L����z�z�'�%��a��0�1�1���	�"����s�A2�2B�Bc��g}t�}tjd|xsd�}t|�D�]�\}}tjd|�}g}tj|�D]R}|j||j�|j�d|jd�j�d���Ttj|�D]�}|jd�jd�}	|	|vr�(|j|	�||j�|j�|	|jd�j�d�}
|j|
�|j|	jd����t j|�D]o}|jd�jd�}	|	|vs|	|vr�,|j|	�|j||j�|j�|	dd���q���|S)	agReturn a list of citation dicts found in the answer.

    Each dict is ``{idx, start, end, url, meta, prepared}`` where ``idx`` is the
    paragraph index in ``answer`` split on blank lines, ``start``/``end`` are
    offsets inside that paragraph, and ``meta`` is the "(Author, Venue, Year)"
    text (or None for a bare URL). Duplicate URLs are ignored.
    �
\s*\n\s*\n\s*r+�meta)�idxr8r9�urlr=r?�.,;:]rN)�setr.�split�	enumerate�_CODE_FENCE_REr/�_EMPTY_CITE_RE�finditer�appendr8r9�groupr0�_META_RE�rstrip�add�
_PLAIN_URL_RE)�answer�	citations�seen�parasr>�para�stripped�
structured�mr?�items           r(�extract_citationsrVds����I��5�D��H�H�%�v�|��4�E��u�%�&�	��T�!�%�%�b�$�/���
��(�(��2�		�A�
���������u�u�w�������-�-�/��
�		��"�"�8�,�
	3�A��'�'�%�.�'�'��0�C��d�{���H�H�S�M�������u�u�w�������-�-�/��D�
���T�"����c�j�j��1�2�
	3��'�'��1�	�A��'�'�!�*�#�#�G�,�C��d�{�c�Z�/���H�H�S�M����������u�u�w����
�	�7&�N��c�d�	ddlm}||xsd�}|jjd�xsd}|jj�xs||g}|jr|jd|j���dj|�S#t$r|xsdjd�cYSwxYw)zoNormalize a URL for equality checks: lowercase host, drop fragment and
    trailing slash, keep path and query.r)�urlsplitr+�/�?)
�urllib.parserY�pathrJ�netloc�lower�queryrG�joinr)r?rY�pr]�partss     r(�	_norm_urlrd�s���
'�)��S�Y�B����v�v�}�}�S�!�(�S�������!�(�S�$�/���7�7��L�L�1�Q�W�W�I��'��w�w�u�~����'��	�r�!�!�#�&�&�'�s�BB�B/�.B/c	�@�t�}tj5ttjj|i�jdg��}ddd�D�].}t
|t�s�|jd�dk(r�|jdd�}|r|jt|��tj|jdd�xsd�D]:}|jt|jd�jd	����<��|jd
g�xsgD]X}t
|t�s�|jd�xs|jd�xsd}|s�?|jt|���Z��1|S#1swY��@xYw)a4All URLs the research agent actually opened or saw in search results.

    Rebuilt from the task's ``_search_details`` (every ``web_search`` result
    URL plus every ``fetch_page`` URL and any link inside fetched content). A
    citation that is NOT in here was never grounded by the agent's own tools.
    �_search_detailsN�tool�
fetch_pager?r+rrr@�results�link)rAr�
_data_lock�list�tasksr�
isinstance�dictrKrdrLrFrHrJ)�task_id�urls�details�entry�urTr&s       r(�_retrieved_urlsru�sW���5�D�	
���L��q�w�w�{�{�7�B�/�3�3�4E�r�J�K��L��+���%��&���9�9�V���,��	�	�%��$�A������1��&�"�+�+�E�I�I�i��,D�,J��K�
@������1�7�7�1�:�#4�#4�W�#=�>�?�
@�����9�b�)�/�R�	+�A��!�T�"��E�E�%�L�7�A�E�E�&�M�7�R����H�H�Y�q�\�*�		+�+��K�#L�L�s�:F�Fc��	tjtj|��}t
|�}|jdg�xsgD]N}t|t�s�|jd�|jd�fD]}|s�t
|�|k(s�y�Py#t$r}td|�d|���Yd}~yd}~wwxYw)	z�Existence probe for a source the agent never retrieved: re-search the
    URL itself and require a same-URL hit. Used before ANY invention verdict.z$[critic] existence probe failed for �: NFrir?rjT)
rr3r�
web_searchrrrdrrnro)r?�resr'�targetr&rts      r(�_citation_existsr{�s�����j�j����c�*�+���s�^�F�
�W�W�Y��
#�
)�r� ���a����e�e�E�l�A�E�E�&�M�2�
 ����1���/��
 � �
����
�4�S�E��A�3�?�@����s�(B�	C�&B<�<Cc�D�	tjtj||d���}|jd�r	d||dd�Sd||jd	|�|jd
d�|jdd�xsdd
�S#t$r}d|d|��d�cYd}~Sd}~wwxYw)zFFetch a single source page; never raises. ``ok=False`` on any failure.r)�	max_chars�chunkFzfetch failed: )�okr?�errorNr�Tr?�titler+r)rr?�	final_urlr�r)rr3rrhrr)r?r}�pager's    r(�
_fetch_sourcer��s���H��z�z�!�,�,�s�i�q�I�J���x�x����C�$�w�-�@�@����X�X�e�S�)����'�2�&��8�8�I�r�*�0�b����	�H��C�N�1�#�2F�G�G��H�s�+B�	B�B�B�Bc���|r|jd�s!dddd|xsijd�xsdd�dfS|jdd	�xsd	}|j�s
d
d
dddd�dfSd|d
�d|jd�xsd�d|jd�xs|jd�xsd	�d|jdd	��d|dtj��
}t	t
||�}|�yt
|�}|�y|dfS)z?Single critic call for one citation. Returns (verdict, failed).rF�absentNr�zsource could not be fetched)�exists�url_ok�
claim_support�corrected_meta�reasonrr+Tz4page content unreachable (likely JS-only or blocked)zCLAIMED CITATION:
URL: r?z
Metadata: r=z(none given)z

CLAIM CONTEXT:
�preparedrQz

FETCHED SOURCE (title: r�z):
)NT)rr0r�VERIFY_FETCH_CHARSr)�_VERIFY_SYSTEMr:)�cit�srcr"�bodyrr6�verdicts       r(�_verify_sourcer��s?���C�G�G�D�M���%�"��y�b�o�o�g�.�O�2O�
��
�	��7�7�9�b�!�'�R�D��:�:�<���%�"�L�
��
�	���E�
�|���W�W�V�_�6��7�8��7�7�:�.�G�#�'�'�&�/�G�R�H�I"�"%�'�'�'�2�"6�!7�t��%��%�%�&�
'�	)�	��n�d�D�9�D��|���T�"�G�����E�>�rWc�,���sy�jd�dus�jd�dury|r4|jd�r#|jd�xsdj�sy	�jd
d�}|dvryt�fd
�dD��ry|dk(ry	y)z7Map a critic verdict (+ fetch info) to an action token.�	UNCHECKEDr�Fr��UNVERIFIABLErrr+�	AMBIGUOUSr�r�)�unsupportedr�c3�D�K�|]}�j|�du���y�w)FN)r)�.0�fr�s  �r(�	<genexpr>zclassify.<locals>.<genexpr>s�����
Q�q�7�;�;�q�>�U�"�
Q�s� )�	author_ok�venue_ok�year_ok�METADATA_FIX�partial�KEEP)rr0�any)r�r��supports`  r(�classifyr�s�������{�{�8���%����X�)>�%�)G���C�G�G�D�M�3�7�7�9�+=�+C��*J�*J�*L���k�k�/�8�4�G��+�+��
�
Q�,P�
Q�Q���)���rWc���d}	ddlm}||�j}|xs|�d|��j	�}	tjtj|��}|jdg�}|D]q}|jdd�}d}		ddlm}||�j}	|r|	|k(s�8|jd�s�Jd||jd	d�|jdd�d
�cS|D]L}|jd�s�d|jdd�|jd	d�|jdd�d
�cSd|d�S#t$rY��,wxYw#t$ri}Y��wxYw#t$rY��wxYw)
zATargeted re-search for a corrected source; prefers the same host.r+r)�urlparse� rir?�full_contentT�
page_title)rr?r�rF)rr?)
r\r�r^rr0rr3rrxr)
r?r=�hostr�r`ryrir&rt�r_hosts
          r(�_search_and_fetchr�s���
�D�
�)���}�#�#���{�s�m�1�S�E�"�(�(�*�E���j�j����e�,�-���g�g�i��$�G�
�
r��
�E�E�%������	�-��a�[�'�'�F���$��A�E�E�.�,A��q�1�5�5��r�3J�WX�W\�W\�]k�mo�Wp�q�q�
r��A���5�5�� ��q�u�u�U�B�'7�!�%�%��VX�BY�fg�fk�fk�lz�|~�f�A�
A�A���$�$��-�
��
��
�������	��	�s4�D<�(E�
E�<	E	�E	�E�E�	E)�(E)c�j�t|d�}t|||�\}}|ryt||�}|dk(r�|xsijd�s�d}t	tdtj��D]b}t|d|jd�xsd�}|jd	�s�7t|||�\}	}
|
s|	s�L|	jd�s�^|	}|}n|r|}ni|�ddi�}|d
k(r�|r�|r|jd	�s�t	tdtj��D]o}t|d|jd�xsd�}|jd	�s�7t|||�\}	}
|
r�J|	s�M|	jd�s�_|	}|}t|	|�}n|dk(rP|xsijd�}|jd�}|r|rd|�d
|d�d�}
d|�d�}n_|rd|d�d�}
d}nQd}
d}nL|d
k(rd}
|xsijd�xsd}n+|dk(r"d}
d}|xsijd�}|rd|�d�}nd}
d}|||
|fS)zEFull per-citation pipeline. Returns (action, note, replace, verdict).r?)r�u#critic unavailable — not verifiedNNr�r�Nrr=r+rr�r��(z) [�]zmetadata corrected to (�)z(Author, Venue, uncertain) [u4metadata could not be confirmed — marked uncertainu?bare source URL without metadata — could not confirm metadatar�u2source could not be confirmed — citation removedr�u>claim only partially supported, or sources conflict — reviewzmetadata uncertain (u) — review�verified)	r�r�r�rr�maxr�VERIFY_RETRIESr�)r�r"r�r��failed�action�best�_�src2�v2�f2�	corrected�claimed_meta�replace�note�_maybes                r(�_verify_oner�=s���
��E�
�
#�C�$�S�#�t�4�O�G�V�
���g�s�
#�F�
�����B�(;�(;�<L�(M����s�1�a�.�.�/�0�
	�A�$�S��Z������1F�B�G�D��8�8�D�>��#�C��t�4�F�B������v�v�&�'������
	���G�9��9�"2�D�9�G�
���G�S�S�W�W�T�]��s�1�a�.�.�/�0�		�A�$�S��Z������1F�B�G�D��8�8�D�>��#�C��t�4�F�B���"�����!1�����!�"�d�+���		�����]��'�'�(8�9�	��w�w�v������)��C��E�
�|�1�5�G�,�Y�K�q�9�D�
�4�S��Z�L��B�G�I�D��G�T�D�	�>�	!����
�2�"�"�8�,�d�0d��	�;�	���O���-�R�$�$�%5�6���)�&���>�D������4��'�)�)rWc�,�|sydddddd�}tg}|D]\}|jdd	�}|j|d
�}|jd�rd|d��nd}|j|�d
|d�|����^|jd�dj|�S)Nr+u✓u△u✗u⚠u∅)r�r�r�r�r�r�r�u•r�u — r�r?z
</details>�
)�_TAGLINErrGra)�verdicts�marks�lines�vr��symbolr�s       r(�_build_verification_blockr��s����������
�E��J�E�
�3�����x��(�����6�5�)��&'�e�e�F�m��q��y�k�"���
����x�q��5��
�4�&�1�2�	3�

�L�L����9�9�U��rWc��tj5tjj|�xsi}ddd�g}|rjd�s||fSt	j
d|�}t
|�}|s||fSt|�}ddlm	}	|	�}
tjtjd|xsd��D]-}|
|jd�jd�xxd	z
cc<�/|D�cic]!}|ds�	|d|
j|dd���#}
}t!t#td
d��}i}|D�]8}|dt%|�kr||dnd}||d
<d}d}|dsd\}}n|d|vrt'|d�sd\}}|
j|dd�}||krdnd|�d|�d�}|�||j)�ddf\}}}}nt+||�\}}}}|r|dk7r	|xsd|z}|j-|d|jd�|||r|xsijd�nd|r|xsijd�ndd��|���|j/|dg�j-|d|d|d����;|r�|j1�D]y\}}||}g}d}t3|d���D]2}|j-|||d�|j-|d�|d}�4|j-||d�dj5|�||<�{dj5|�}t7|�}|r|j�|z}||fS#1swY��xYwcc}w)z�Extract citations, verify each, patch the answer, return (final, verdicts).

    Research tasks only; non-research answers and citation-free answers pass
    through untouched. Never raises.
    N�researchr<r)�Counterr+r?r@r�VERIFY_MAX_CITES_PER_URL�r>r�)r�u.citation has no URL to verify — cannot exist)r�ubURL was never retrieved by research and no verification search could find it — likely fabricatedz SAME SOURCE CITED z TIMES for u, separate claims — verify each maps to it.r�r=r�r�)r?r=r�r�r�r�r8r9)r8r9r�c��|dS)Nr8�)�xs r(�<lambda>z"run_verification.<locals>.<lambda>�s
���7��rW)�keyr�z

)rrkrmrr.rBrVru�collectionsr�rIrFrDr/rHrJ�int�getattr�lenr{r0r�rG�
setdefault�items�sortedrar�) rp�sidrMr"�tr�rPrN�	retrievedr��usagerT�c�per_cite_usage�	max_cites�per_parar�rQ�
pre_action�pre_note�over�	over_noter�r�r�r�r>�editsrc�posr'�blocks                                 r(�run_verificationr��s���
���'�
�G�G�K�K�� �&�B��'��H�����z�*��x����H�H�%�v�.�E�!�&�)�I���x��� ��(�I�#�

�I�E�
�
�
�~�1�1�"�f�l��C�
D�3��
�a�g�g�e�n�#�#�G�,�-��2�-�3�@I�V�1�Q�u�X�a��h��	�	�!�E�(�A� 6�6�V�N�V��G�A�9�1�=�>�I��H��'��$'��J��U��$;�u�S��Z� �����J���
����5�z�$� �J����Z�y�
(�#�C��J�/�(�$�
�H��!�!�#�e�*�a�0���	�)��/��v�[���Fr�s�	��!�-7����9I�2�t�-S�*�F�D�'�7�-8��d�-C�*�F�D�'�7���>�1��J�B�)�+�D�����u�:��G�G�F�O���GN�w�}�"�1�1�2B�C�TX�7>�w�}�"�)�)�(�3�D�

�	�������E�
�B�/�6�6��W���5�z�"�8�
�G'�R�"�.�.�*�		(�J�C����:�D��E��C��E�';�<�
�����T�#�a��j�1�2����Q�y�\�*���h��
�
�L�L��c�d��$������E�#�J�		(����U�#��%�h�/�E������5�(���8���k'�'��0Ws�$L;�6
M�M�;Mc��tj�}	t||||�\}}tj5tjj|�}|r)||d<t
tj�|z
d�|d<ddd�tj||||�y#1swY�"xYw#t$r�}	td|�d|	���tj5tjj|�}|r
ddd	|	��d
�g|d<ddd�n#1swYnxYwtj||||�Yd}	~	yd}	~	wwxYw)z�Thread-pool entry point called from the orchestration event loop.

    Guarantees the task always finalizes: the (possibly patched) answer on
    success, the original answer untouched on any failure.
    �
_verificationr�_verification_durationNz+[critic] verification pass failed for task rwr[r�zverification pass failed: )r?r�r�)
r r�rrkrmr�round�_finalize_taskrr)
rpr�rMr�r"�started�finalr��ttr's
          r(�run_verification_workerr��s1���i�i�k�G�5�*�7�C���F���x�
�\�\�	O������W�%�B��&.��?�#�/4�T�Y�Y�[�7�5J�A�/N��+�,�		O�
	
����#�u�d�3�	O�	O���5�
�;�G�9�B�q�c�J�K�
�\�\�	S������W�%�B��/2�k�2L�Q�C�0P�(R�'S��?�#�	S�	S�	S��
	
����#�v�t�4�4��5�sG�!B/�AB#� B/�#B,�(B/�/	D?�8!D:�/D�	D:�D	�D:�:D?)�gpuiX)ip)r�)�__doc__rr.r r�server.features.staterr��compilerIrErL�DOTALLrDr�r)r:rVrdrur{r�r�r�r�r�r�r�r�r�rWr(�<module>r�s���� �	���#�
P��"�2�:�:�R�������P�Q����
�
�<�=�
�����L�"�)�)�4��B���<�2�j
'� �8�"�"!�H�$%�@C*�L�([�|5rW