📄 context.cpython-312.pyc
/home/palash/git/local-ai/server/features/__pycache__/context.cpython-312.pyc
Language: pyc • Lines: 432
�

�Y�j-���dZddlZddlZddlZddlZddlZddlmZd�Zd�Z	dd�Z
d�Zdd�Zdd	�Z
d
�Zd�Zd�Zd
�Zd�Zdd�Zd�Zd�Zy)z:Token estimation, context trimming and context compaction.�N)�Mc��tjdd|tj��}tjdd|tj��}tjdd|�}tjdd|�j�}|S)Nz<style[^>]*>.*?</style>�)�flagsz<script[^>]*>.*?</script>z<[^>]+>� z\s+)�re�sub�DOTALL�strip)�texts �server/features/context.py�
strip_htmlr
sf��
�6�6�,�b�$�b�i�i�H�D�
�6�6�.��D��	�	�J�D�
�6�6�*�c�4�(�D�
�6�6�&�#�t�$�*�*�,�D��K�c��|sytd�|D��}|t|�dzkDrdnd}tt|�|z�S)Nrc3�>K�|]}t|�dkDs�d���y�w)��N)�ord)�.0�chs  r
�	<genexpr>z_text_tokens.<locals>.<genexpr>s����4�"�S��W�t�^�A�4�s��g333333�?g@g@)�sum�len�int)�s�	non_ascii�divisors   r
�_text_tokensrsB�����4��4�4�I���Q��$��.�c�C�G��s�1�v��� � rc	�l�|rtjnd}|D�]}|tjz
}|jdd�}t	|t
�r|t
|�z
}n�t	|t�r}|D]x}t	|t�s�|jd�}|dk(r|t
|jdd��z
}�I|dvr|tjz
}�a|dvs�f|tjz
}�z|jd�xsgD]#}|t
tj|��z
}�%��td	|�S)
Nr�contentr�typer)�	image_url�input_image�image��	audio_url�input_audio�audio�
tool_callsr)r�TOOLS_TOKEN_COST�PER_MESSAGE_OVERHEAD�get�
isinstance�strr�list�dict�IMAGE_TOKEN_COST�AUDIO_TOKEN_COST�json�dumps�max)�messages�
include_tools�total�msgr �part�ptype�tcs        r
�estimate_tokensr=s(��"/�A���Q�E��2��
��'�'�'���'�'�)�R�(���g�s�#��\�'�*�*�E����
&��	
0��!�$��-������(���F�?��\�$�(�(�6�2�*>�?�?�E��C�C��Q�/�/�/�E��C�C��Q�/�/�/�E�	
0��'�'�,�'�-�2�	2�B��\�$�*�*�R�.�1�1�E�	2�-2�4�q�%�=�rc�h�t|�}d}|r(|djd�dk(r|jd�}t|�tj
kDrJt
|�dkDr<|jd�t|�tj
kDrt
|�dkDr�<|r|jd|�|S)Nr�role�systemr)r/r,�popr=r�MAX_INPUT_TOKENSr�insert)r6�trimmed�sys_msgs   r
�trim_messages_for_contextrF>s����8�n�G��G��7�1�:�>�>�&�)�X�5��+�+�a�.��
�'�
"�Q�%7�%7�
7�C��L�1�<L����A���'�
"�Q�%7�%7�
7�C��L�1�<L�����q�'�"��Nrc�j�tj|�ddd�d|d�gdddd�}	tj|�tjtj
|�|d	�
�}|j
�|j�ddd
dS#t$r}td|���Yd}~yd}~wwxYw)Nr@zsYou summarize conversations concisely, preserving key facts, decisions, user preferences, and unresolved questions.�r?r �userig333333�?F)�modelr6�
max_tokens�temperature�stream�x)r3�timeout�choicesr�messager z$[compact] LLM summarization failed: )
r�server_model_id�mark_slot_kv_dirty�requests�post�
server_url�raise_for_statusr3�	Exception�print)r�mode�payload�r�es     r
�_summarize_with_llmr^Js����"�"�4�(�!�Q�
���-�
�����G��	���T�"��M�M�!�,�,�t�,�7�C�H��	�����v�v�x�	�"�1�%�i�0��;�;����
�4�Q�C�8�9����s�A+B�	B2�B-�-B2c�0�t|�}d}|r(|djd�dk(r|jd�}t|�|dzkr
|r|g|zS|S|dkDr|d|n|}|dkDr||dng}d}|D]�}|jdd�}	|jdd�}
t	|
t�rag}|
D]I}t	|t
�s�|jd	�d
k(s�)|j
|jd
d���Kdj|�}
|
s��|d|	�d
|
�d�z
}��|j�s
|r|g|zS|Stjd|��|�}
|
�
|r|g|zS|Sg}|r|j
|�|j
dd|
��d��|j|�|S)z�Return a compacted COPY of the message list (summary + recent messages)
    WITHOUT modifying the stored session. Old messages are summarized, not deleted.Nrr?r@rr�unknownr r!rr�[z]: z

z\Compress the following conversation into a short paragraph, keeping all important details:

z[Compressed context]: rH)r/r,rArr-r0�append�joinrrr^�extend)r6�
keep_messagesrZ�msgsrE�
to_compact�recent�compact_text�mr?r �parts�p�summary�new_msgss               r
�compact_messages_copyrobs�����>�D��G���Q����F�#�x�/��(�(�1�+��
�4�y�M�A�%�%�%,��	�D� �6�$�6�*7�!�*;��o�
�~�&��J�&3�a�&7�T�=�.�/�
"�R�F��L�
�3���u�u�V�Y�'���%�%�	�2�&���g�t�$��E��
8���a��&��u�u�V�}��.����Q�U�U�6�2�%6�7�
8��h�h�u�o�G����!�D�6��W�I�T�2�2��3�����%,��	�D� �6�$�6��#�#�
h�iu�hv�w���G���%,��	�D� �6�$�6��H������ ��O�O�X�4J�7�)�2T�U�V��O�O�F���Orc�^�g}|D]�}|jd�}t|t�s|j|��6g}d}|D]:}t|t�s�|jd�dvrd}�*|j|��<|r|jddd��|ji|�d|i����|S)	aReturn a COPY of ``messages`` with content parts the LLM backend cannot
    process (e.g. ``audio_url``) removed, so a stale multimodal message can't
    make llama-server reject the whole request (HTTP 400 "unsupported
    content[].type"). The stored session is left untouched.
    r Fr!r%TruF[Voice message omitted — audio input is not supported by this model]�r!r)r,r-r/rbr0)r6�	sanitizedr9r rk�
dropped_audiorls       r
�sanitize_content_for_llmrt�s����I��4���'�'�)�$���'�4�(����S�!�����
��	�A��a��&���u�u�V�}� E�E� $�
���L�L��O�
	���L�L�"�d��
�	���2�C�2��E�2�3�+4�,�rc���|syd}|jd�ratjj|j	dd�d�}tjjtj|�}n`|jd�rO|td�dj	dd�d}tjjtj|�}|rtjj|�sy|S)zResolve a ``/uploads/`` or ``/output/`` URL to a local file path, or
    ``None`` if the URL is unknown or the file is missing.Nz	/uploads/�?rrz/output/)�
startswith�os�path�basename�splitrcr�UPLOADS_DIRr�IMG_PATH�isfile)�url�fpath�fname�rels    r
�resolve_image_pathr��s������E�
�~�~�k�"���� � ����3��!2�1�!5�6�������Q�]�]�E�2��	���
�	#��#�j�/�"�#�)�)�#�q�1�!�4�������Q�Z�Z��-��������u�-���Lrc��|sy|jd�r|St|�}|sytjj	|�dj�}dddddd�j
|d�}	t|d	�5}|j�}ddd�d|�d
tj�j���S#1swY�2xYw#t$rYywxYw)aResolve an image URL the server can serve to a ``data:`` URL.

    Understands ``/uploads/`` (user uploads) and ``/output/`` (generated
    images) so those bytes can be embedded in the LLM request. ``data:`` URLs
    pass through unchanged; anything unknown returns ``None``.
    Nzdata:rz	image/pngz
image/jpegz
image/webpz	image/gif)z.pngz.jpgz.jpegz.webpz.gif�rbz;base64,)
rwr�rxry�splitext�lowerr,�open�read�OSError�base64�	b64encode�decode)rr��ext�mime�f�raws      r
�_image_to_data_urlr��s�����
�~�~�g���
��s�#�E���
�'�'�
�
�5�
!�!�
$�
*�
*�
,�C�������
�c�#�|��
	��
�%��
�	�!��&�&�(�C�	��4�&���!1�!1�#�!6�!=�!=�!?� @�A�A�		�	������s*�.C�:B<�C�<C�C�	C�Cc��d}t|�D]\}}|jd�dk(s�|}�g}t|�D�]\}}|jd�}t|t�s|j	|��:g}|D]�}t|t
�r�|jd�dk(r}|jdi�jdd�}||k(r=t
|�}	|	r|j	dd|	id	���r|j	d
d|�d�d
����|j	d
d|�d�d
����|j	|���|j	i|�d|i����|S)u�Return a COPY of ``messages`` where ``image_url`` parts are kept lean.

    The image attached to the most recent user message stays visible (its bytes
    are embedded via ``_image_to_data_url``), preserving the current turn's
    vision. Every older image becomes a compact text marker carrying the file
    URL — the model can call ``read_image`` to actually view one, so history
    never drags megabytes of base64 through the context window.
    �����r?rIr r!r"rr)r!r"rz[IMAGE: �]rqu0 — use the read_image tool to view this image])�	enumerater,r-r/rbr0r�)
r6�
last_user_idx�irj�outr rkrlr�data_urls
          r
�_reference_historical_imagesr��s^���M��(�#����1��5�5��=�F�"��M��
�C��(�#�,���1��%�%�	�"���'�4�(��J�J�q�M�����	 �A��!�T�"�q�u�u�V�}��'C��e�e�K��,�0�0���;���
�%�1�#�6�H�����k��PX�HY�%Z�[� ��L�L�&�H�S�E��:K�!L�M��L�L�$*�&.�s�e�3c�$d������Q��#	 �$	�
�
�*�a�*��E�*�+�1,�2�Jrc�@�t|�D]{}|jd�dk7r�|jd�}t|t�s�:	t	j
|�}|jd�dus�d|jd�s�v|dcSy#ttf$rY��wxYw)zpReturn the image URL of the most recent successful ``read_image`` tool
    result in ``messages`` (or ``None``).r?�toolr �okTr"N)�reversedr,r-r.r3�loads�	TypeError�
ValueError)r6rjr �datas    r
�_latest_read_image_urlr�s����h�
�%���5�5��=�F�"���%�%�	�"���'�3�'��	��:�:�g�&�D��8�8�D�>�T�!�d�h�h�{�&;���$�$�%���	�:�&�	��	�s�B�B�Bc	�*�t|�}t|�}t|�}|tjkrFt|�}tj5tjj|d�ddd�|Std|�d|�dtj�d��t||��}t|�}tdt|����tj5|tj|<ddd�|S#1swY|SxYw#1swY|SxYw)u`Build the message list to send to the LLM. When the conversation nears the
    context limit, old messages are summarized into a compressed context block —
    but the stored session is left untouched, so no messages are deleted.
    Historical images are referenced by path (see ``read_image``) instead of
    being re-sent as base64 on every round.Nz[context] Session z
 estimate z tokens exceeds threshold z%; building compressed context for LLM)rZz4[context] Compressed context built; estimate after: )rtr�r=r�AUTO_COMPACT_THRESHOLDrF�_effective_contexts_lock�_effective_contextsrArYro)�sidr6rZr8�context�	compacteds      r
�prepare_context_for_llmr�s��(��1�H�+�H�5�H��H�%�E���(�(�(�+�H�5��
�
'�
'�	1�
�!�!�%�%�c�4�0�	1���	��s�e�:�e�W�4N�q�Og�Og�Nh�iN�O�P�%�h�T�:�I�'�	�2�G�	�@��QX�AY�@Z�
[�\�	
�	#�	#�-�%,����c�"�-��N�	1����
-��N�s�!C;�D�;D�Dc��tj5tjj|�}ddd��t	|�St	|�S#1swY�!xYw)z�Report the token count the UI shows: the compressed context actually sent
    to the LLM once compression has kicked in, falling back to the full history.N)rr�r�r,r=)r�r6�cacheds   r
�effective_token_estimater�.sT��
�	#�	#�0��&�&�*�*�3�/��0�
���v�&�&��8�$�$�	0�0�s� A�Ac�B�t||�}t|�}||||kDd�S)z�Token report for the UI: effective count sent to the LLM, the raw stored
    count, and whether context compression is currently active.)�token_estimate�raw_token_estimate�context_compressed)r�r=)r�r6�	effectiver�s    r
�context_token_reportr�8s1��)��h�7�I�
�(�
#�C�#�!�!�I�o��r)T)�gpu)�r�)�__doc__r�r3rxrrT�server.features.staterrrr=rFr^rortr�r�r�r�r�r�r��rr
�<module>r�se��@�
��	�	��#��!��@	��0&�R�@�"B�<(�V�$�.%�	r