pyc • Lines: 432�
�Y�j- � � � d Z ddlZddlZddlZddlZddlZddlmZ d� Zd� Z dd�Z
d� Zdd�Zdd �Z
d
� Zd� Zd� Zd
� Zd� Zdd�Zd� Zd� Zy)z:Token estimation, context trimming and context compaction.� N)�Mc � � t j dd| t j �� } t j dd| t j �� } t j dd| � } t j dd| � j � } | S )Nz<style[^>]*>.*?</style>� )�flagsz<script[^>]*>.*?</script>z<[^>]+>� z\s+)�re�sub�DOTALL�strip)�texts �server/features/context.py�
strip_htmlr
sf � �
�6�6�,�b�$�b�i�i�H�D�
�6�6�.��D�� � �J�D�
�6�6�*�c�4�(�D�
�6�6�&�#�t�$�*�*�,�D��K� c � � | syt d� | D � � }|t | � dz kD rdnd}t t | � |z � S )Nr c 3 �>