2025-05-16 01:40:55,678 - __main__ - INFO - Got --pdfs argument, going to add to the work queue 2025-05-16 01:40:55,679 - __main__ - INFO - Loading file at tests/gnarly_pdfs/horribleocr.pdf as PDF document 2025-05-16 01:40:55,679 - __main__ - INFO - Found 1 total pdf paths to add 2025-05-16 01:40:55,683 - __main__ - INFO - Calculated items_per_group: 500 based on average pages per PDF: 1.00 2025-05-16 02:00:47,526 - __main__ - INFO - Got --pdfs argument, going to add to the work queue 2025-05-16 02:00:47,526 - __main__ - INFO - Loading file at tests/gnarly_pdfs/horribleocr.pdf as PDF document 2025-05-16 02:00:47,526 - __main__ - INFO - Found 1 total pdf paths to add 2025-05-16 02:00:47,529 - __main__ - INFO - Calculated items_per_group: 500 based on average pages per PDF: 1.00 2025-05-16 02:00:47,720 - __main__ - INFO - Starting pipeline with PID 347737 2025-05-16 02:00:47,720 - __main__ - INFO - Downloading model with hugging face 'allenai/olmOCR-7B-0225-preview' 2025-05-16 02:03:44,171 - __main__ - INFO - Got --pdfs argument, going to add to the work queue 2025-05-16 02:03:44,171 - __main__ - INFO - Loading file at tests/gnarly_pdfs/horribleocr.pdf as PDF document 2025-05-16 02:03:44,171 - __main__ - INFO - Found 1 total pdf paths to add 2025-05-16 02:03:44,175 - __main__ - INFO - Calculated items_per_group: 500 based on average pages per PDF: 1.00 2025-05-16 02:03:44,416 - __main__ - INFO - Starting pipeline with PID 347855 2025-05-16 02:03:44,416 - __main__ - INFO - Downloading model with hugging face 'allenai/olmOCR-7B-0225-preview' 2025-05-16 02:06:11,039 - __main__ - INFO - Got --pdfs argument, going to add to the work queue 2025-05-16 02:06:11,039 - __main__ - INFO - Loading file at tests/gnarly_pdfs/horribleocr.pdf as PDF document 2025-05-16 02:06:11,039 - __main__ - INFO - Found 1 total pdf paths to add 2025-05-16 02:06:11,043 - __main__ - INFO - Calculated items_per_group: 500 based on average pages per PDF: 1.00 2025-05-16 02:06:11,311 - __main__ - INFO - Starting pipeline with PID 347960 2025-05-16 02:06:11,311 - __main__ - INFO - Downloading model with hugging face 'allenai/olmOCR-7B-0225-preview' 2025-05-17 01:34:19,419 - __main__ - INFO - Got --pdfs argument, going to add to the work queue 2025-05-17 01:34:19,419 - __main__ - INFO - Loading file at tests/gnarly_pdfs/horribleocr.pdf as PDF document 2025-05-17 01:34:19,420 - __main__ - INFO - Found 1 total pdf paths to add 2025-05-17 01:34:19,424 - __main__ - INFO - Calculated items_per_group: 500 based on average pages per PDF: 1.00 2025-05-17 01:34:19,659 - __main__ - INFO - Starting pipeline with PID 370510 2025-05-17 01:34:19,659 - __main__ - INFO - Downloading model with hugging face 'allenai/olmOCR-7B-0225-preview' 2025-05-17 01:42:18,000 - __main__ - INFO - Got --pdfs argument, going to add to the work queue 2025-05-17 01:42:18,000 - __main__ - INFO - Loading file at tests/gnarly_pdfs/horribleocr.pdf as PDF document 2025-05-17 01:42:18,000 - __main__ - INFO - Found 1 total pdf paths to add 2025-05-17 01:42:18,004 - __main__ - INFO - Calculated items_per_group: 500 based on average pages per PDF: 1.00 2025-05-17 01:42:18,204 - __main__ - INFO - Starting pipeline with PID 370697 2025-05-17 01:42:18,204 - __main__ - INFO - Downloading model with hugging face 'allenai/olmOCR-7B-0225-preview' 2025-05-17 01:46:11,794 - __main__ - WARNING - Attempt 1: Please wait for sglang server to become ready... 2025-05-17 01:46:12,829 - __main__ - WARNING - Attempt 2: Please wait for sglang server to become ready... 2025-05-17 01:46:13,879 - __main__ - WARNING - Attempt 3: Please wait for sglang server to become ready... 2025-05-17 01:46:14,944 - __main__ - WARNING - Attempt 4: Please wait for sglang server to become ready... 2025-05-17 01:46:16,011 - __main__ - WARNING - Attempt 5: Please wait for sglang server to become ready... 2025-05-17 01:46:17,040 - __main__ - WARNING - Attempt 6: Please wait for sglang server to become ready... 2025-05-17 01:46:17,815 - sglang - INFO - [2025-05-17 01:46:17] server_args=ServerArgs(model_path='allenai/olmOCR-7B-0225-preview', tokenizer_path='allenai/olmOCR-7B-0225-preview', tokenizer_mode='auto', load_format='auto', trust_remote_code=False, dtype='auto', kv_cache_dtype='auto', quantization_param_path=None, quantization=None, context_length=None, device='cuda', served_model_name='allenai/olmOCR-7B-0225-preview', chat_template='qwen2-vl', is_embedding=False, revision=None, skip_tokenizer_init=False, host='127.0.0.1', port=30024, mem_fraction_static=0.8, max_running_requests=None, max_total_tokens=None, chunked_prefill_size=2048, max_prefill_tokens=16384, schedule_policy='lpm', schedule_conservativeness=1.0, cpu_offload_gb=0, prefill_only_one_req=False, tp_size=1, stream_interval=1, stream_output=False, random_seed=47741023, constrained_json_whitespace_pattern=None, watchdog_timeout=300, download_dir=None, base_gpu_id=0, log_level='info', log_level_http='warning', log_requests=False, show_time_cost=False, enable_metrics=False, decode_log_interval=40, api_key=None, file_storage_pth='sglang_storage', enable_cache_report=False, dp_size=1, load_balance_method='round_robin', ep_size=1, dist_init_addr=None, nnodes=1, node_rank=0, json_model_override_args='{}', lora_paths=None, max_loras_per_batch=8, attention_backend='flashinfer', sampling_backend='flashinfer', grammar_backend='outlines', speculative_draft_model_path=None, speculative_algorithm=None, speculative_num_steps=5, speculative_num_draft_tokens=64, speculative_eagle_topk=8, enable_double_sparsity=False, ds_channel_config_path=None, ds_heavy_channel_num=32, ds_heavy_token_num=256, ds_heavy_channel_type='qk', ds_sparse_decode_threshold=4096, disable_radix_cache=False, disable_jump_forward=False, disable_cuda_graph=False, disable_cuda_graph_padding=False, disable_outlines_disk_cache=False, disable_custom_all_reduce=False, disable_mla=False, disable_overlap_schedule=False, enable_mixed_chunk=False, enable_dp_attention=False, enable_ep_moe=False, enable_torch_compile=False, torch_compile_max_bs=32, cuda_graph_max_bs=8, cuda_graph_bs=None, torchao_config='', enable_nan_detection=False, enable_p2p_check=False, triton_attention_reduce_in_fp32=False, triton_attention_num_kv_splits=8, num_continuous_decode_steps=1, delete_ckpt_after_loading=False, enable_memory_saver=False, allow_auto_truncate=False, enable_custom_logit_processor=False, tool_call_parser=None) 2025-05-17 01:46:17,815 - __main__ - INFO - [2025-05-17 01:46:17] server_args=ServerArgs(model_path='allenai/olmOCR-7B-0225-preview', tokenizer_path='allenai/olmOCR-7B-0225-preview', tokenizer_mode='auto', load_format='auto', trust_remote_code=False, dtype='auto', kv_cache_dtype='auto', quantization_param_path=None, quantization=None, context_length=None, device='cuda', served_model_name='allenai/olmOCR-7B-0225-preview', chat_template='qwen2-vl', is_embedding=False, revision=None, skip_tokenizer_init=False, host='127.0.0.1', port=30024, mem_fraction_static=0.8, max_running_requests=None, max_total_tokens=None, chunked_prefill_size=2048, max_prefill_tokens=16384, schedule_policy='lpm', schedule_conservativeness=1.0, cpu_offload_gb=0, prefill_only_one_req=False, tp_size=1, stream_interval=1, stream_output=False, random_seed=47741023, constrained_json_whitespace_pattern=None, watchdog_timeout=300, download_dir=None, base_gpu_id=0, log_level='info', log_level_http='warning', log_requests=False, show_time_cost=False, enable_metrics=False, decode_log_interval=40, api_key=None, file_storage_pth='sglang_storage', enable_cache_report=False, dp_size=1, load_balance_method='round_robin', ep_size=1, dist_init_addr=None, nnodes=1, node_rank=0, json_model_override_args='{}', lora_paths=None, max_loras_per_batch=8, attention_backend='flashinfer', sampling_backend='flashinfer', grammar_backend='outlines', speculative_draft_model_path=None, speculative_algorithm=None, speculative_num_steps=5, speculative_num_draft_tokens=64, speculative_eagle_topk=8, enable_double_sparsity=False, ds_channel_config_path=None, ds_heavy_channel_num=32, ds_heavy_token_num=256, ds_heavy_channel_type='qk', ds_sparse_decode_threshold=4096, disable_radix_cache=False, disable_jump_forward=False, disable_cuda_graph=False, disable_cuda_graph_padding=False, disable_outlines_disk_cache=False, disable_custom_all_reduce=False, disable_mla=False, disable_overlap_schedule=False, enable_mixed_chunk=False, enable_dp_attention=False, enable_ep_moe=False, enable_torch_compile=False, torch_compile_max_bs=32, cuda_graph_max_bs=8, cuda_graph_bs=None, torchao_config='', enable_nan_detection=False, enable_p2p_check=False, triton_attention_reduce_in_fp32=False, triton_attention_num_kv_splits=8, num_continuous_decode_steps=1, delete_ckpt_after_loading=False, enable_memory_saver=False, allow_auto_truncate=False, enable_custom_logit_processor=False, tool_call_parser=None) 2025-05-17 01:46:18,110 - __main__ - WARNING - Attempt 7: Please wait for sglang server to become ready... 2025-05-17 01:46:19,171 - __main__ - WARNING - Attempt 8: Please wait for sglang server to become ready... 2025-05-17 01:46:20,235 - __main__ - WARNING - Attempt 9: Please wait for sglang server to become ready... 2025-05-17 01:46:21,302 - __main__ - WARNING - Attempt 10: Please wait for sglang server to become ready... 2025-05-17 01:46:22,439 - __main__ - WARNING - Attempt 11: Please wait for sglang server to become ready... 2025-05-17 01:46:23,255 - sglang - INFO - [2025-05-17 01:46:23] Use chat template for the OpenAI-compatible API server: qwen2-vl 2025-05-17 01:46:23,255 - __main__ - INFO - [2025-05-17 01:46:23] Use chat template for the OpenAI-compatible API server: qwen2-vl 2025-05-17 01:46:23,515 - __main__ - WARNING - Attempt 12: Please wait for sglang server to become ready... 2025-05-17 01:46:24,582 - __main__ - WARNING - Attempt 13: Please wait for sglang server to become ready... 2025-05-17 01:46:25,649 - __main__ - WARNING - Attempt 14: Please wait for sglang server to become ready... 2025-05-17 01:46:26,716 - __main__ - WARNING - Attempt 15: Please wait for sglang server to become ready... 2025-05-17 01:46:27,783 - __main__ - WARNING - Attempt 16: Please wait for sglang server to become ready... 2025-05-17 01:46:28,847 - __main__ - WARNING - Attempt 17: Please wait for sglang server to become ready... 2025-05-17 01:46:29,906 - __main__ - WARNING - Attempt 18: Please wait for sglang server to become ready... 2025-05-17 01:46:30,974 - __main__ - WARNING - Attempt 19: Please wait for sglang server to become ready... 2025-05-17 01:46:32,048 - __main__ - WARNING - Attempt 20: Please wait for sglang server to become ready... 2025-05-17 01:46:32,899 - sglang - INFO - [2025-05-17 01:46:32 TP0] Overlap scheduler is disabled for multimodal models. 2025-05-17 01:46:32,900 - __main__ - INFO - [2025-05-17 01:46:32 TP0] Overlap scheduler is disabled for multimodal models. 2025-05-17 01:46:33,125 - __main__ - WARNING - Attempt 21: Please wait for sglang server to become ready... 2025-05-17 01:46:33,730 - sglang - INFO - [2025-05-17 01:46:33 TP0] Automatically reduce --mem-fraction-static to 0.760 because this is a multimodal model. 2025-05-17 01:46:33,730 - __main__ - INFO - [2025-05-17 01:46:33 TP0] Automatically reduce --mem-fraction-static to 0.760 because this is a multimodal model. 2025-05-17 01:46:33,730 - sglang - INFO - [2025-05-17 01:46:33 TP0] Automatically turn off --chunked-prefill-size and disable radix cache for qwen2-vl. 2025-05-17 01:46:33,730 - __main__ - INFO - [2025-05-17 01:46:33 TP0] Automatically turn off --chunked-prefill-size and disable radix cache for qwen2-vl. 2025-05-17 01:46:33,731 - sglang - INFO - [2025-05-17 01:46:33 TP0] Init torch distributed begin. 2025-05-17 01:46:33,731 - __main__ - INFO - [2025-05-17 01:46:33 TP0] Init torch distributed begin. 2025-05-17 01:46:34,202 - __main__ - WARNING - Attempt 22: Please wait for sglang server to become ready... 2025-05-17 01:46:35,274 - __main__ - WARNING - Attempt 23: Please wait for sglang server to become ready... 2025-05-17 01:46:36,341 - __main__ - WARNING - Attempt 24: Please wait for sglang server to become ready... 2025-05-17 01:46:37,409 - __main__ - WARNING - Attempt 25: Please wait for sglang server to become ready... 2025-05-17 01:46:38,477 - __main__ - WARNING - Attempt 26: Please wait for sglang server to become ready... 2025-05-17 01:46:39,122 - sglang - INFO - [2025-05-17 01:46:39 TP0] Load weight begin. avail mem=23.33 GB 2025-05-17 01:46:39,123 - __main__ - INFO - [2025-05-17 01:46:39 TP0] Load weight begin. avail mem=23.33 GB 2025-05-17 01:46:39,555 - __main__ - WARNING - Attempt 27: Please wait for sglang server to become ready... 2025-05-17 01:46:40,623 - __main__ - WARNING - Attempt 28: Please wait for sglang server to become ready... 2025-05-17 01:46:40,752 - sglang - INFO - [2025-05-17 01:46:40 TP0] Using model weights format ['*.safetensors'] 2025-05-17 01:46:40,752 - __main__ - INFO - [2025-05-17 01:46:40 TP0] Using model weights format ['*.safetensors'] 2025-05-17 01:46:41,534 - sglang - INFO - Loading safetensors checkpoint shards: 0% Completed | 0/4 [00:00: Failed to establish a new connection: [Errno 101] Network is unreachable 2025-05-17 22:08:15,386 - __main__ - INFO - urllib3.exceptions.NewConnectionError: : Failed to establish a new connection: [Errno 101] Network is unreachable 2025-05-17 22:08:15,386 - sglang - INFO - 2025-05-17 22:08:15,386 - __main__ - INFO - 2025-05-17 22:08:15,386 - sglang - INFO - The above exception was the direct cause of the following exception: 2025-05-17 22:08:15,386 - __main__ - INFO - The above exception was the direct cause of the following exception: 2025-05-17 22:08:15,386 - sglang - INFO - 2025-05-17 22:08:15,386 - __main__ - INFO - 2025-05-17 22:08:15,386 - sglang - INFO - Traceback (most recent call last): 2025-05-17 22:08:15,386 - __main__ - INFO - Traceback (most recent call last): 2025-05-17 22:08:15,386 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/requests/adapters.py", line 667, in send 2025-05-17 22:08:15,386 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/requests/adapters.py", line 667, in send 2025-05-17 22:08:15,386 - sglang - INFO - resp = conn.urlopen( 2025-05-17 22:08:15,386 - __main__ - INFO - resp = conn.urlopen( 2025-05-17 22:08:15,386 - sglang - INFO - ^^^^^^^^^^^^^ 2025-05-17 22:08:15,386 - __main__ - INFO - ^^^^^^^^^^^^^ 2025-05-17 22:08:15,387 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/urllib3/connectionpool.py", line 841, in urlopen 2025-05-17 22:08:15,387 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/urllib3/connectionpool.py", line 841, in urlopen 2025-05-17 22:08:15,387 - sglang - INFO - retries = retries.increment( 2025-05-17 22:08:15,387 - __main__ - INFO - retries = retries.increment( 2025-05-17 22:08:15,387 - sglang - INFO - ^^^^^^^^^^^^^^^^^^ 2025-05-17 22:08:15,387 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^ 2025-05-17 22:08:15,387 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/urllib3/util/retry.py", line 519, in increment 2025-05-17 22:08:15,387 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/urllib3/util/retry.py", line 519, in increment 2025-05-17 22:08:15,387 - sglang - INFO - raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] 2025-05-17 22:08:15,387 - __main__ - INFO - raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] 2025-05-17 22:08:15,387 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-05-17 22:08:15,387 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-05-17 22:08:15,387 - sglang - INFO - urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='huggingface.co', port=443): Max retries exceeded with url: /api/models/allenai/olmOCR-7B-0225-preview (Caused by NewConnectionError(': Failed to establish a new connection: [Errno 101] Network is unreachable')) 2025-05-17 22:08:15,387 - __main__ - INFO - urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='huggingface.co', port=443): Max retries exceeded with url: /api/models/allenai/olmOCR-7B-0225-preview (Caused by NewConnectionError(': Failed to establish a new connection: [Errno 101] Network is unreachable')) 2025-05-17 22:08:15,387 - sglang - INFO - 2025-05-17 22:08:15,387 - __main__ - INFO - 2025-05-17 22:08:15,387 - sglang - INFO - During handling of the above exception, another exception occurred: 2025-05-17 22:08:15,387 - __main__ - INFO - During handling of the above exception, another exception occurred: 2025-05-17 22:08:15,387 - sglang - INFO - 2025-05-17 22:08:15,387 - __main__ - INFO - 2025-05-17 22:08:15,387 - sglang - INFO - Traceback (most recent call last): 2025-05-17 22:08:15,387 - __main__ - INFO - Traceback (most recent call last): 2025-05-17 22:08:15,387 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/managers/scheduler.py", line 1773, in run_scheduler_process 2025-05-17 22:08:15,387 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/managers/scheduler.py", line 1773, in run_scheduler_process 2025-05-17 22:08:15,387 - sglang - INFO - scheduler = Scheduler(server_args, port_args, gpu_id, tp_rank, dp_rank) 2025-05-17 22:08:15,387 - __main__ - INFO - scheduler = Scheduler(server_args, port_args, gpu_id, tp_rank, dp_rank) 2025-05-17 22:08:15,387 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-05-17 22:08:15,387 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-05-17 22:08:15,387 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/managers/scheduler.py", line 239, in __init__ 2025-05-17 22:08:15,387 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/managers/scheduler.py", line 239, in __init__ 2025-05-17 22:08:15,387 - sglang - INFO - self.tp_worker = TpWorkerClass( 2025-05-17 22:08:15,387 - __main__ - INFO - self.tp_worker = TpWorkerClass( 2025-05-17 22:08:15,388 - sglang - INFO - ^^^^^^^^^^^^^^ 2025-05-17 22:08:15,388 - __main__ - INFO - ^^^^^^^^^^^^^^ 2025-05-17 22:08:15,388 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/managers/tp_worker.py", line 68, in __init__ 2025-05-17 22:08:15,388 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/managers/tp_worker.py", line 68, in __init__ 2025-05-17 22:08:15,388 - sglang - INFO - self.model_runner = ModelRunner( 2025-05-17 22:08:15,388 - __main__ - INFO - self.model_runner = ModelRunner( 2025-05-17 22:08:15,388 - sglang - INFO - ^^^^^^^^^^^^ 2025-05-17 22:08:15,388 - __main__ - INFO - ^^^^^^^^^^^^ 2025-05-17 22:08:15,388 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/model_executor/model_runner.py", line 185, in __init__ 2025-05-17 22:08:15,388 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/model_executor/model_runner.py", line 185, in __init__ 2025-05-17 22:08:15,388 - sglang - INFO - self.load_model() 2025-05-17 22:08:15,388 - __main__ - INFO - self.load_model() 2025-05-17 22:08:15,388 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/model_executor/model_runner.py", line 306, in load_model 2025-05-17 22:08:15,388 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/model_executor/model_runner.py", line 306, in load_model 2025-05-17 22:08:15,388 - sglang - INFO - self.model = get_model( 2025-05-17 22:08:15,388 - __main__ - INFO - self.model = get_model( 2025-05-17 22:08:15,388 - sglang - INFO - ^^^^^^^^^^ 2025-05-17 22:08:15,388 - __main__ - INFO - ^^^^^^^^^^ 2025-05-17 22:08:15,388 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/model_loader/__init__.py", line 22, in get_model 2025-05-17 22:08:15,388 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/model_loader/__init__.py", line 22, in get_model 2025-05-17 22:08:15,388 - sglang - INFO - return loader.load_model( 2025-05-17 22:08:15,388 - __main__ - INFO - return loader.load_model( 2025-05-17 22:08:15,388 - sglang - INFO - ^^^^^^^^^^^^^^^^^^ 2025-05-17 22:08:15,388 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^ 2025-05-17 22:08:15,388 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/model_loader/loader.py", line 362, in load_model 2025-05-17 22:08:15,388 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/model_loader/loader.py", line 362, in load_model 2025-05-17 22:08:15,388 - sglang - INFO - model.load_weights(self._get_all_weights(model_config, model)) 2025-05-17 22:08:15,388 - __main__ - INFO - model.load_weights(self._get_all_weights(model_config, model)) 2025-05-17 22:08:15,388 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/models/qwen2_vl.py", line 576, in load_weights 2025-05-17 22:08:15,388 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/models/qwen2_vl.py", line 576, in load_weights 2025-05-17 22:08:15,388 - sglang - INFO - for name, loaded_weight in weights: 2025-05-17 22:08:15,388 - __main__ - INFO - for name, loaded_weight in weights: 2025-05-17 22:08:15,389 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/model_loader/loader.py", line 335, in _get_all_weights 2025-05-17 22:08:15,389 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/model_loader/loader.py", line 335, in _get_all_weights 2025-05-17 22:08:15,389 - sglang - INFO - yield from self._get_weights_iterator(primary_weights) 2025-05-17 22:08:15,389 - __main__ - INFO - yield from self._get_weights_iterator(primary_weights) 2025-05-17 22:08:15,389 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-05-17 22:08:15,389 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-05-17 22:08:15,389 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/model_loader/loader.py", line 303, in _get_weights_iterator 2025-05-17 22:08:15,389 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/model_loader/loader.py", line 303, in _get_weights_iterator 2025-05-17 22:08:15,389 - sglang - INFO - hf_folder, hf_weights_files, use_safetensors = self._prepare_weights( 2025-05-17 22:08:15,389 - __main__ - INFO - hf_folder, hf_weights_files, use_safetensors = self._prepare_weights( 2025-05-17 22:08:15,389 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^ 2025-05-17 22:08:15,389 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^^^^^ 2025-05-17 22:08:15,389 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/model_loader/loader.py", line 255, in _prepare_weights 2025-05-17 22:08:15,389 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/model_loader/loader.py", line 255, in _prepare_weights 2025-05-17 22:08:15,389 - sglang - INFO - hf_folder = download_weights_from_hf( 2025-05-17 22:08:15,389 - __main__ - INFO - hf_folder = download_weights_from_hf( 2025-05-17 22:08:15,389 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-05-17 22:08:15,389 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-05-17 22:08:15,389 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/model_loader/weight_utils.py", line 246, in download_weights_from_hf 2025-05-17 22:08:15,389 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/model_loader/weight_utils.py", line 246, in download_weights_from_hf 2025-05-17 22:08:15,389 - sglang - INFO - file_list = fs.ls(model_name_or_path, detail=False, revision=revision) 2025-05-17 22:08:15,389 - __main__ - INFO - file_list = fs.ls(model_name_or_path, detail=False, revision=revision) 2025-05-17 22:08:15,389 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-05-17 22:08:15,389 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-05-17 22:08:15,389 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/huggingface_hub/hf_file_system.py", line 368, in ls 2025-05-17 22:08:15,389 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/huggingface_hub/hf_file_system.py", line 368, in ls 2025-05-17 22:08:15,389 - sglang - INFO - resolved_path = self.resolve_path(path, revision=revision) 2025-05-17 22:08:15,389 - __main__ - INFO - resolved_path = self.resolve_path(path, revision=revision) 2025-05-17 22:08:15,389 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-05-17 22:08:15,389 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-05-17 22:08:15,389 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/huggingface_hub/hf_file_system.py", line 209, in resolve_path 2025-05-17 22:08:15,389 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/huggingface_hub/hf_file_system.py", line 209, in resolve_path 2025-05-17 22:08:15,389 - sglang - INFO - repo_and_revision_exist, err = self._repo_and_revision_exist(repo_type, repo_id, revision) 2025-05-17 22:08:15,390 - __main__ - INFO - repo_and_revision_exist, err = self._repo_and_revision_exist(repo_type, repo_id, revision) 2025-05-17 22:08:15,390 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-05-17 22:08:15,390 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-05-17 22:08:15,390 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/huggingface_hub/hf_file_system.py", line 125, in _repo_and_revision_exist 2025-05-17 22:08:15,390 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/huggingface_hub/hf_file_system.py", line 125, in _repo_and_revision_exist 2025-05-17 22:08:15,390 - sglang - INFO - self._api.repo_info( 2025-05-17 22:08:15,390 - __main__ - INFO - self._api.repo_info( 2025-05-17 22:08:15,390 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/huggingface_hub/utils/_validators.py", line 114, in _inner_fn 2025-05-17 22:08:15,390 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/huggingface_hub/utils/_validators.py", line 114, in _inner_fn 2025-05-17 22:08:15,390 - sglang - INFO - return fn(*args, **kwargs) 2025-05-17 22:08:15,390 - __main__ - INFO - return fn(*args, **kwargs) 2025-05-17 22:08:15,390 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^ 2025-05-17 22:08:15,390 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^^ 2025-05-17 22:08:15,390 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/huggingface_hub/hf_api.py", line 2816, in repo_info 2025-05-17 22:08:15,390 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/huggingface_hub/hf_api.py", line 2816, in repo_info 2025-05-17 22:08:15,390 - sglang - INFO - return method( 2025-05-17 22:08:15,390 - __main__ - INFO - return method( 2025-05-17 22:08:15,390 - sglang - INFO - ^^^^^^^ 2025-05-17 22:08:15,390 - __main__ - INFO - ^^^^^^^ 2025-05-17 22:08:15,390 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/huggingface_hub/utils/_validators.py", line 114, in _inner_fn 2025-05-17 22:08:15,390 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/huggingface_hub/utils/_validators.py", line 114, in _inner_fn 2025-05-17 22:08:15,390 - sglang - INFO - return fn(*args, **kwargs) 2025-05-17 22:08:15,390 - __main__ - INFO - return fn(*args, **kwargs) 2025-05-17 22:08:15,390 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^ 2025-05-17 22:08:15,390 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^^ 2025-05-17 22:08:15,390 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/huggingface_hub/hf_api.py", line 2600, in model_info 2025-05-17 22:08:15,390 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/huggingface_hub/hf_api.py", line 2600, in model_info 2025-05-17 22:08:15,390 - sglang - INFO - r = get_session().get(path, headers=headers, timeout=timeout, params=params) 2025-05-17 22:08:15,390 - __main__ - INFO - r = get_session().get(path, headers=headers, timeout=timeout, params=params) 2025-05-17 22:08:15,390 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-05-17 22:08:15,390 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-05-17 22:08:15,390 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/requests/sessions.py", line 602, in get 2025-05-17 22:08:15,391 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/requests/sessions.py", line 602, in get 2025-05-17 22:08:15,391 - sglang - INFO - return self.request("GET", url, **kwargs) 2025-05-17 22:08:15,391 - __main__ - INFO - return self.request("GET", url, **kwargs) 2025-05-17 22:08:15,391 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-05-17 22:08:15,391 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-05-17 22:08:15,391 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/requests/sessions.py", line 589, in request 2025-05-17 22:08:15,391 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/requests/sessions.py", line 589, in request 2025-05-17 22:08:15,391 - sglang - INFO - resp = self.send(prep, **send_kwargs) 2025-05-17 22:08:15,391 - __main__ - INFO - resp = self.send(prep, **send_kwargs) 2025-05-17 22:08:15,391 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-05-17 22:08:15,391 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-05-17 22:08:15,391 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/requests/sessions.py", line 703, in send 2025-05-17 22:08:15,391 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/requests/sessions.py", line 703, in send 2025-05-17 22:08:15,391 - sglang - INFO - r = adapter.send(request, **kwargs) 2025-05-17 22:08:15,391 - __main__ - INFO - r = adapter.send(request, **kwargs) 2025-05-17 22:08:15,391 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-05-17 22:08:15,391 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-05-17 22:08:15,391 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/huggingface_hub/utils/_http.py", line 96, in send 2025-05-17 22:08:15,391 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/huggingface_hub/utils/_http.py", line 96, in send 2025-05-17 22:08:15,391 - sglang - INFO - return super().send(request, *args, **kwargs) 2025-05-17 22:08:15,391 - __main__ - INFO - return super().send(request, *args, **kwargs) 2025-05-17 22:08:15,392 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-05-17 22:08:15,392 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-05-17 22:08:15,392 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/requests/adapters.py", line 700, in send 2025-05-17 22:08:15,392 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/requests/adapters.py", line 700, in send 2025-05-17 22:08:15,392 - sglang - INFO - raise ConnectionError(e, request=request) 2025-05-17 22:08:15,392 - __main__ - INFO - raise ConnectionError(e, request=request) 2025-05-17 22:08:15,392 - sglang - INFO - requests.exceptions.ConnectionError: (MaxRetryError("HTTPSConnectionPool(host='huggingface.co', port=443): Max retries exceeded with url: /api/models/allenai/olmOCR-7B-0225-preview (Caused by NewConnectionError(': Failed to establish a new connection: [Errno 101] Network is unreachable'))"), '(Request ID: ae2f99d1-1701-487a-8bb1-775aa4b85868)') 2025-05-17 22:08:15,392 - __main__ - INFO - requests.exceptions.ConnectionError: (MaxRetryError("HTTPSConnectionPool(host='huggingface.co', port=443): Max retries exceeded with url: /api/models/allenai/olmOCR-7B-0225-preview (Caused by NewConnectionError(': Failed to establish a new connection: [Errno 101] Network is unreachable'))"), '(Request ID: ae2f99d1-1701-487a-8bb1-775aa4b85868)') 2025-05-17 22:08:15,392 - sglang - INFO - 2025-05-17 22:08:15,392 - __main__ - INFO - 2025-05-17 22:08:15,393 - sglang - INFO - [2025-05-17 22:08:15] Received sigquit from a child proces. It usually means the child failed. 2025-05-17 22:08:15,393 - __main__ - INFO - [2025-05-17 22:08:15] Received sigquit from a child proces. It usually means the child failed. 2025-05-17 22:08:15,556 - __main__ - WARNING - SGLang server task ended 2025-05-17 22:08:16,284 - __main__ - WARNING - Attempt 26: Please wait for sglang server to become ready... 2025-05-17 22:08:17,351 - __main__ - WARNING - Attempt 27: Please wait for sglang server to become ready... 2025-05-17 22:08:18,418 - __main__ - WARNING - Attempt 28: Please wait for sglang server to become ready... 2025-05-17 22:08:19,486 - __main__ - WARNING - Attempt 29: Please wait for sglang server to become ready... 2025-05-17 22:08:20,554 - __main__ - WARNING - Attempt 30: Please wait for sglang server to become ready... 2025-05-17 22:08:21,617 - __main__ - WARNING - Attempt 31: Please wait for sglang server to become ready... 2025-05-17 22:08:21,653 - sglang - INFO - [2025-05-17 22:08:21] server_args=ServerArgs(model_path='allenai/olmOCR-7B-0225-preview', tokenizer_path='allenai/olmOCR-7B-0225-preview', tokenizer_mode='auto', load_format='auto', trust_remote_code=False, dtype='auto', kv_cache_dtype='auto', quantization_param_path=None, quantization=None, context_length=None, device='cuda', served_model_name='allenai/olmOCR-7B-0225-preview', chat_template='qwen2-vl', is_embedding=False, revision=None, skip_tokenizer_init=False, host='127.0.0.1', port=30024, mem_fraction_static=0.8, max_running_requests=None, max_total_tokens=None, chunked_prefill_size=2048, max_prefill_tokens=16384, schedule_policy='lpm', schedule_conservativeness=1.0, cpu_offload_gb=0, prefill_only_one_req=False, tp_size=1, stream_interval=1, stream_output=False, random_seed=245788542, constrained_json_whitespace_pattern=None, watchdog_timeout=300, download_dir=None, base_gpu_id=0, log_level='info', log_level_http='warning', log_requests=False, show_time_cost=False, enable_metrics=False, decode_log_interval=40, api_key=None, file_storage_pth='sglang_storage', enable_cache_report=False, dp_size=1, load_balance_method='round_robin', ep_size=1, dist_init_addr=None, nnodes=1, node_rank=0, json_model_override_args='{}', lora_paths=None, max_loras_per_batch=8, attention_backend='flashinfer', sampling_backend='flashinfer', grammar_backend='outlines', speculative_draft_model_path=None, speculative_algorithm=None, speculative_num_steps=5, speculative_num_draft_tokens=64, speculative_eagle_topk=8, enable_double_sparsity=False, ds_channel_config_path=None, ds_heavy_channel_num=32, ds_heavy_token_num=256, ds_heavy_channel_type='qk', ds_sparse_decode_threshold=4096, disable_radix_cache=False, disable_jump_forward=False, disable_cuda_graph=False, disable_cuda_graph_padding=False, disable_outlines_disk_cache=False, disable_custom_all_reduce=False, disable_mla=False, disable_overlap_schedule=False, enable_mixed_chunk=False, enable_dp_attention=False, enable_ep_moe=False, enable_torch_compile=False, torch_compile_max_bs=32, cuda_graph_max_bs=8, cuda_graph_bs=None, torchao_config='', enable_nan_detection=False, enable_p2p_check=False, triton_attention_reduce_in_fp32=False, triton_attention_num_kv_splits=8, num_continuous_decode_steps=1, delete_ckpt_after_loading=False, enable_memory_saver=False, allow_auto_truncate=False, enable_custom_logit_processor=False, tool_call_parser=None) 2025-05-17 22:08:21,653 - __main__ - INFO - [2025-05-17 22:08:21] server_args=ServerArgs(model_path='allenai/olmOCR-7B-0225-preview', tokenizer_path='allenai/olmOCR-7B-0225-preview', tokenizer_mode='auto', load_format='auto', trust_remote_code=False, dtype='auto', kv_cache_dtype='auto', quantization_param_path=None, quantization=None, context_length=None, device='cuda', served_model_name='allenai/olmOCR-7B-0225-preview', chat_template='qwen2-vl', is_embedding=False, revision=None, skip_tokenizer_init=False, host='127.0.0.1', port=30024, mem_fraction_static=0.8, max_running_requests=None, max_total_tokens=None, chunked_prefill_size=2048, max_prefill_tokens=16384, schedule_policy='lpm', schedule_conservativeness=1.0, cpu_offload_gb=0, prefill_only_one_req=False, tp_size=1, stream_interval=1, stream_output=False, random_seed=245788542, constrained_json_whitespace_pattern=None, watchdog_timeout=300, download_dir=None, base_gpu_id=0, log_level='info', log_level_http='warning', log_requests=False, show_time_cost=False, enable_metrics=False, decode_log_interval=40, api_key=None, file_storage_pth='sglang_storage', enable_cache_report=False, dp_size=1, load_balance_method='round_robin', ep_size=1, dist_init_addr=None, nnodes=1, node_rank=0, json_model_override_args='{}', lora_paths=None, max_loras_per_batch=8, attention_backend='flashinfer', sampling_backend='flashinfer', grammar_backend='outlines', speculative_draft_model_path=None, speculative_algorithm=None, speculative_num_steps=5, speculative_num_draft_tokens=64, speculative_eagle_topk=8, enable_double_sparsity=False, ds_channel_config_path=None, ds_heavy_channel_num=32, ds_heavy_token_num=256, ds_heavy_channel_type='qk', ds_sparse_decode_threshold=4096, disable_radix_cache=False, disable_jump_forward=False, disable_cuda_graph=False, disable_cuda_graph_padding=False, disable_outlines_disk_cache=False, disable_custom_all_reduce=False, disable_mla=False, disable_overlap_schedule=False, enable_mixed_chunk=False, enable_dp_attention=False, enable_ep_moe=False, enable_torch_compile=False, torch_compile_max_bs=32, cuda_graph_max_bs=8, cuda_graph_bs=None, torchao_config='', enable_nan_detection=False, enable_p2p_check=False, triton_attention_reduce_in_fp32=False, triton_attention_num_kv_splits=8, num_continuous_decode_steps=1, delete_ckpt_after_loading=False, enable_memory_saver=False, allow_auto_truncate=False, enable_custom_logit_processor=False, tool_call_parser=None) 2025-05-17 22:08:22,679 - __main__ - WARNING - Attempt 32: Please wait for sglang server to become ready... 2025-05-17 22:08:23,765 - __main__ - WARNING - Attempt 33: Please wait for sglang server to become ready... 2025-05-17 22:08:24,823 - __main__ - WARNING - Attempt 34: Please wait for sglang server to become ready... 2025-05-17 22:08:25,890 - __main__ - WARNING - Attempt 35: Please wait for sglang server to become ready... 2025-05-17 22:08:27,034 - __main__ - WARNING - Attempt 36: Please wait for sglang server to become ready... 2025-05-17 22:08:28,099 - __main__ - WARNING - Attempt 37: Please wait for sglang server to become ready... 2025-05-17 22:08:28,528 - sglang - INFO - [2025-05-17 22:08:28] Use chat template for the OpenAI-compatible API server: qwen2-vl 2025-05-17 22:08:28,528 - __main__ - INFO - [2025-05-17 22:08:28] Use chat template for the OpenAI-compatible API server: qwen2-vl 2025-05-17 22:08:29,176 - __main__ - WARNING - Attempt 38: Please wait for sglang server to become ready... 2025-05-17 22:08:30,243 - __main__ - WARNING - Attempt 39: Please wait for sglang server to become ready... 2025-05-17 22:08:31,310 - __main__ - WARNING - Attempt 40: Please wait for sglang server to become ready... 2025-05-17 22:08:32,378 - __main__ - WARNING - Attempt 41: Please wait for sglang server to become ready... 2025-05-17 22:08:33,446 - __main__ - WARNING - Attempt 42: Please wait for sglang server to become ready... 2025-05-17 22:08:34,428 - sglang - INFO - [2025-05-17 22:08:34 TP0] Overlap scheduler is disabled for multimodal models. 2025-05-17 22:08:34,428 - __main__ - INFO - [2025-05-17 22:08:34 TP0] Overlap scheduler is disabled for multimodal models. 2025-05-17 22:08:34,522 - __main__ - WARNING - Attempt 43: Please wait for sglang server to become ready... 2025-05-17 22:08:34,610 - sglang - INFO - [2025-05-17 22:08:34 TP0] Automatically reduce --mem-fraction-static to 0.760 because this is a multimodal model. 2025-05-17 22:08:34,610 - __main__ - INFO - [2025-05-17 22:08:34 TP0] Automatically reduce --mem-fraction-static to 0.760 because this is a multimodal model. 2025-05-17 22:08:34,610 - sglang - INFO - [2025-05-17 22:08:34 TP0] Automatically turn off --chunked-prefill-size and disable radix cache for qwen2-vl. 2025-05-17 22:08:34,610 - __main__ - INFO - [2025-05-17 22:08:34 TP0] Automatically turn off --chunked-prefill-size and disable radix cache for qwen2-vl. 2025-05-17 22:08:34,610 - sglang - INFO - [2025-05-17 22:08:34 TP0] Init torch distributed begin. 2025-05-17 22:08:34,611 - __main__ - INFO - [2025-05-17 22:08:34 TP0] Init torch distributed begin. 2025-05-17 22:08:35,599 - __main__ - WARNING - Attempt 44: Please wait for sglang server to become ready... 2025-05-17 22:08:36,667 - __main__ - WARNING - Attempt 45: Please wait for sglang server to become ready... 2025-05-17 22:08:37,722 - __main__ - WARNING - Attempt 46: Please wait for sglang server to become ready... 2025-05-17 22:08:38,789 - __main__ - WARNING - Attempt 47: Please wait for sglang server to become ready... 2025-05-17 22:08:39,857 - __main__ - WARNING - Attempt 48: Please wait for sglang server to become ready... 2025-05-17 22:08:39,992 - sglang - INFO - [2025-05-17 22:08:39 TP0] Load weight begin. avail mem=23.33 GB 2025-05-17 22:08:39,992 - __main__ - INFO - [2025-05-17 22:08:39 TP0] Load weight begin. avail mem=23.33 GB 2025-05-17 22:08:40,668 - sglang - INFO - [2025-05-17 22:08:40 TP0] Scheduler hit an exception: Traceback (most recent call last): 2025-05-17 22:08:40,668 - __main__ - INFO - [2025-05-17 22:08:40 TP0] Scheduler hit an exception: Traceback (most recent call last): 2025-05-17 22:08:40,669 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/urllib3/connection.py", line 198, in _new_conn 2025-05-17 22:08:40,669 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/urllib3/connection.py", line 198, in _new_conn 2025-05-17 22:08:40,669 - sglang - INFO - sock = connection.create_connection( 2025-05-17 22:08:40,669 - __main__ - INFO - sock = connection.create_connection( 2025-05-17 22:08:40,669 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-05-17 22:08:40,669 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-05-17 22:08:40,669 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/urllib3/util/connection.py", line 85, in create_connection 2025-05-17 22:08:40,669 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/urllib3/util/connection.py", line 85, in create_connection 2025-05-17 22:08:40,669 - sglang - INFO - raise err 2025-05-17 22:08:40,669 - __main__ - INFO - raise err 2025-05-17 22:08:40,669 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/urllib3/util/connection.py", line 73, in create_connection 2025-05-17 22:08:40,669 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/urllib3/util/connection.py", line 73, in create_connection 2025-05-17 22:08:40,669 - sglang - INFO - sock.connect(sa) 2025-05-17 22:08:40,669 - __main__ - INFO - sock.connect(sa) 2025-05-17 22:08:40,670 - sglang - INFO - OSError: [Errno 101] Network is unreachable 2025-05-17 22:08:40,670 - __main__ - INFO - OSError: [Errno 101] Network is unreachable 2025-05-17 22:08:40,670 - sglang - INFO - 2025-05-17 22:08:40,670 - __main__ - INFO - 2025-05-17 22:08:40,670 - sglang - INFO - The above exception was the direct cause of the following exception: 2025-05-17 22:08:40,670 - __main__ - INFO - The above exception was the direct cause of the following exception: 2025-05-17 22:08:40,670 - sglang - INFO - 2025-05-17 22:08:40,670 - __main__ - INFO - 2025-05-17 22:08:40,670 - sglang - INFO - Traceback (most recent call last): 2025-05-17 22:08:40,670 - __main__ - INFO - Traceback (most recent call last): 2025-05-17 22:08:40,670 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/urllib3/connectionpool.py", line 787, in urlopen 2025-05-17 22:08:40,670 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/urllib3/connectionpool.py", line 787, in urlopen 2025-05-17 22:08:40,670 - sglang - INFO - response = self._make_request( 2025-05-17 22:08:40,670 - __main__ - INFO - response = self._make_request( 2025-05-17 22:08:40,670 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^ 2025-05-17 22:08:40,670 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^^ 2025-05-17 22:08:40,671 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/urllib3/connectionpool.py", line 488, in _make_request 2025-05-17 22:08:40,671 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/urllib3/connectionpool.py", line 488, in _make_request 2025-05-17 22:08:40,671 - sglang - INFO - raise new_e 2025-05-17 22:08:40,671 - __main__ - INFO - raise new_e 2025-05-17 22:08:40,671 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/urllib3/connectionpool.py", line 464, in _make_request 2025-05-17 22:08:40,671 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/urllib3/connectionpool.py", line 464, in _make_request 2025-05-17 22:08:40,671 - sglang - INFO - self._validate_conn(conn) 2025-05-17 22:08:40,671 - __main__ - INFO - self._validate_conn(conn) 2025-05-17 22:08:40,671 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/urllib3/connectionpool.py", line 1093, in _validate_conn 2025-05-17 22:08:40,671 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/urllib3/connectionpool.py", line 1093, in _validate_conn 2025-05-17 22:08:40,671 - sglang - INFO - conn.connect() 2025-05-17 22:08:40,671 - __main__ - INFO - conn.connect() 2025-05-17 22:08:40,671 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/urllib3/connection.py", line 704, in connect 2025-05-17 22:08:40,671 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/urllib3/connection.py", line 704, in connect 2025-05-17 22:08:40,671 - sglang - INFO - self.sock = sock = self._new_conn() 2025-05-17 22:08:40,671 - __main__ - INFO - self.sock = sock = self._new_conn() 2025-05-17 22:08:40,671 - sglang - INFO - ^^^^^^^^^^^^^^^^ 2025-05-17 22:08:40,672 - __main__ - INFO - ^^^^^^^^^^^^^^^^ 2025-05-17 22:08:40,672 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/urllib3/connection.py", line 213, in _new_conn 2025-05-17 22:08:40,672 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/urllib3/connection.py", line 213, in _new_conn 2025-05-17 22:08:40,672 - sglang - INFO - raise NewConnectionError( 2025-05-17 22:08:40,672 - __main__ - INFO - raise NewConnectionError( 2025-05-17 22:08:40,672 - sglang - INFO - urllib3.exceptions.NewConnectionError: : Failed to establish a new connection: [Errno 101] Network is unreachable 2025-05-17 22:08:40,672 - __main__ - INFO - urllib3.exceptions.NewConnectionError: : Failed to establish a new connection: [Errno 101] Network is unreachable 2025-05-17 22:08:40,672 - sglang - INFO - 2025-05-17 22:08:40,672 - __main__ - INFO - 2025-05-17 22:08:40,672 - sglang - INFO - The above exception was the direct cause of the following exception: 2025-05-17 22:08:40,672 - __main__ - INFO - The above exception was the direct cause of the following exception: 2025-05-17 22:08:40,672 - sglang - INFO - 2025-05-17 22:08:40,672 - __main__ - INFO - 2025-05-17 22:08:40,672 - sglang - INFO - Traceback (most recent call last): 2025-05-17 22:08:40,672 - __main__ - INFO - Traceback (most recent call last): 2025-05-17 22:08:40,672 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/requests/adapters.py", line 667, in send 2025-05-17 22:08:40,672 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/requests/adapters.py", line 667, in send 2025-05-17 22:08:40,673 - sglang - INFO - resp = conn.urlopen( 2025-05-17 22:08:40,673 - __main__ - INFO - resp = conn.urlopen( 2025-05-17 22:08:40,673 - sglang - INFO - ^^^^^^^^^^^^^ 2025-05-17 22:08:40,673 - __main__ - INFO - ^^^^^^^^^^^^^ 2025-05-17 22:08:40,673 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/urllib3/connectionpool.py", line 841, in urlopen 2025-05-17 22:08:40,673 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/urllib3/connectionpool.py", line 841, in urlopen 2025-05-17 22:08:40,673 - sglang - INFO - retries = retries.increment( 2025-05-17 22:08:40,673 - __main__ - INFO - retries = retries.increment( 2025-05-17 22:08:40,673 - sglang - INFO - ^^^^^^^^^^^^^^^^^^ 2025-05-17 22:08:40,673 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^ 2025-05-17 22:08:40,673 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/urllib3/util/retry.py", line 519, in increment 2025-05-17 22:08:40,673 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/urllib3/util/retry.py", line 519, in increment 2025-05-17 22:08:40,673 - sglang - INFO - raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] 2025-05-17 22:08:40,673 - __main__ - INFO - raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] 2025-05-17 22:08:40,673 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-05-17 22:08:40,673 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-05-17 22:08:40,674 - sglang - INFO - urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='huggingface.co', port=443): Max retries exceeded with url: /api/models/allenai/olmOCR-7B-0225-preview (Caused by NewConnectionError(': Failed to establish a new connection: [Errno 101] Network is unreachable')) 2025-05-17 22:08:40,674 - __main__ - INFO - urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='huggingface.co', port=443): Max retries exceeded with url: /api/models/allenai/olmOCR-7B-0225-preview (Caused by NewConnectionError(': Failed to establish a new connection: [Errno 101] Network is unreachable')) 2025-05-17 22:08:40,674 - sglang - INFO - 2025-05-17 22:08:40,674 - __main__ - INFO - 2025-05-17 22:08:40,674 - sglang - INFO - During handling of the above exception, another exception occurred: 2025-05-17 22:08:40,674 - __main__ - INFO - During handling of the above exception, another exception occurred: 2025-05-17 22:08:40,674 - sglang - INFO - 2025-05-17 22:08:40,674 - __main__ - INFO - 2025-05-17 22:08:40,674 - sglang - INFO - Traceback (most recent call last): 2025-05-17 22:08:40,674 - __main__ - INFO - Traceback (most recent call last): 2025-05-17 22:08:40,674 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/managers/scheduler.py", line 1773, in run_scheduler_process 2025-05-17 22:08:40,674 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/managers/scheduler.py", line 1773, in run_scheduler_process 2025-05-17 22:08:40,674 - sglang - INFO - scheduler = Scheduler(server_args, port_args, gpu_id, tp_rank, dp_rank) 2025-05-17 22:08:40,674 - __main__ - INFO - scheduler = Scheduler(server_args, port_args, gpu_id, tp_rank, dp_rank) 2025-05-17 22:08:40,674 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-05-17 22:08:40,674 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-05-17 22:08:40,674 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/managers/scheduler.py", line 239, in __init__ 2025-05-17 22:08:40,675 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/managers/scheduler.py", line 239, in __init__ 2025-05-17 22:08:40,675 - sglang - INFO - self.tp_worker = TpWorkerClass( 2025-05-17 22:08:40,675 - __main__ - INFO - self.tp_worker = TpWorkerClass( 2025-05-17 22:08:40,675 - sglang - INFO - ^^^^^^^^^^^^^^ 2025-05-17 22:08:40,675 - __main__ - INFO - ^^^^^^^^^^^^^^ 2025-05-17 22:08:40,675 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/managers/tp_worker.py", line 68, in __init__ 2025-05-17 22:08:40,675 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/managers/tp_worker.py", line 68, in __init__ 2025-05-17 22:08:40,675 - sglang - INFO - self.model_runner = ModelRunner( 2025-05-17 22:08:40,675 - __main__ - INFO - self.model_runner = ModelRunner( 2025-05-17 22:08:40,675 - sglang - INFO - ^^^^^^^^^^^^ 2025-05-17 22:08:40,675 - __main__ - INFO - ^^^^^^^^^^^^ 2025-05-17 22:08:40,675 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/model_executor/model_runner.py", line 185, in __init__ 2025-05-17 22:08:40,675 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/model_executor/model_runner.py", line 185, in __init__ 2025-05-17 22:08:40,675 - sglang - INFO - self.load_model() 2025-05-17 22:08:40,675 - __main__ - INFO - self.load_model() 2025-05-17 22:08:40,675 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/model_executor/model_runner.py", line 306, in load_model 2025-05-17 22:08:40,675 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/model_executor/model_runner.py", line 306, in load_model 2025-05-17 22:08:40,676 - sglang - INFO - self.model = get_model( 2025-05-17 22:08:40,676 - __main__ - INFO - self.model = get_model( 2025-05-17 22:08:40,676 - sglang - INFO - ^^^^^^^^^^ 2025-05-17 22:08:40,676 - __main__ - INFO - ^^^^^^^^^^ 2025-05-17 22:08:40,676 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/model_loader/__init__.py", line 22, in get_model 2025-05-17 22:08:40,676 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/model_loader/__init__.py", line 22, in get_model 2025-05-17 22:08:40,676 - sglang - INFO - return loader.load_model( 2025-05-17 22:08:40,676 - __main__ - INFO - return loader.load_model( 2025-05-17 22:08:40,676 - sglang - INFO - ^^^^^^^^^^^^^^^^^^ 2025-05-17 22:08:40,676 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^ 2025-05-17 22:08:40,676 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/model_loader/loader.py", line 362, in load_model 2025-05-17 22:08:40,676 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/model_loader/loader.py", line 362, in load_model 2025-05-17 22:08:40,676 - sglang - INFO - model.load_weights(self._get_all_weights(model_config, model)) 2025-05-17 22:08:40,676 - __main__ - INFO - model.load_weights(self._get_all_weights(model_config, model)) 2025-05-17 22:08:40,676 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/models/qwen2_vl.py", line 576, in load_weights 2025-05-17 22:08:40,676 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/models/qwen2_vl.py", line 576, in load_weights 2025-05-17 22:08:40,677 - sglang - INFO - for name, loaded_weight in weights: 2025-05-17 22:08:40,677 - __main__ - INFO - for name, loaded_weight in weights: 2025-05-17 22:08:40,677 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/model_loader/loader.py", line 335, in _get_all_weights 2025-05-17 22:08:40,677 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/model_loader/loader.py", line 335, in _get_all_weights 2025-05-17 22:08:40,677 - sglang - INFO - yield from self._get_weights_iterator(primary_weights) 2025-05-17 22:08:40,677 - __main__ - INFO - yield from self._get_weights_iterator(primary_weights) 2025-05-17 22:08:40,677 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-05-17 22:08:40,677 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-05-17 22:08:40,677 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/model_loader/loader.py", line 303, in _get_weights_iterator 2025-05-17 22:08:40,677 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/model_loader/loader.py", line 303, in _get_weights_iterator 2025-05-17 22:08:40,677 - sglang - INFO - hf_folder, hf_weights_files, use_safetensors = self._prepare_weights( 2025-05-17 22:08:40,677 - __main__ - INFO - hf_folder, hf_weights_files, use_safetensors = self._prepare_weights( 2025-05-17 22:08:40,677 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^ 2025-05-17 22:08:40,677 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^^^^^ 2025-05-17 22:08:40,677 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/model_loader/loader.py", line 255, in _prepare_weights 2025-05-17 22:08:40,677 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/model_loader/loader.py", line 255, in _prepare_weights 2025-05-17 22:08:40,677 - sglang - INFO - hf_folder = download_weights_from_hf( 2025-05-17 22:08:40,678 - __main__ - INFO - hf_folder = download_weights_from_hf( 2025-05-17 22:08:40,678 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-05-17 22:08:40,678 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-05-17 22:08:40,678 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/model_loader/weight_utils.py", line 246, in download_weights_from_hf 2025-05-17 22:08:40,678 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/model_loader/weight_utils.py", line 246, in download_weights_from_hf 2025-05-17 22:08:40,678 - sglang - INFO - file_list = fs.ls(model_name_or_path, detail=False, revision=revision) 2025-05-17 22:08:40,678 - __main__ - INFO - file_list = fs.ls(model_name_or_path, detail=False, revision=revision) 2025-05-17 22:08:40,678 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-05-17 22:08:40,678 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-05-17 22:08:40,678 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/huggingface_hub/hf_file_system.py", line 368, in ls 2025-05-17 22:08:40,678 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/huggingface_hub/hf_file_system.py", line 368, in ls 2025-05-17 22:08:40,678 - sglang - INFO - resolved_path = self.resolve_path(path, revision=revision) 2025-05-17 22:08:40,678 - __main__ - INFO - resolved_path = self.resolve_path(path, revision=revision) 2025-05-17 22:08:40,678 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-05-17 22:08:40,678 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-05-17 22:08:40,678 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/huggingface_hub/hf_file_system.py", line 209, in resolve_path 2025-05-17 22:08:40,678 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/huggingface_hub/hf_file_system.py", line 209, in resolve_path 2025-05-17 22:08:40,678 - sglang - INFO - repo_and_revision_exist, err = self._repo_and_revision_exist(repo_type, repo_id, revision) 2025-05-17 22:08:40,678 - __main__ - INFO - repo_and_revision_exist, err = self._repo_and_revision_exist(repo_type, repo_id, revision) 2025-05-17 22:08:40,678 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-05-17 22:08:40,678 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-05-17 22:08:40,678 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/huggingface_hub/hf_file_system.py", line 125, in _repo_and_revision_exist 2025-05-17 22:08:40,678 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/huggingface_hub/hf_file_system.py", line 125, in _repo_and_revision_exist 2025-05-17 22:08:40,678 - sglang - INFO - self._api.repo_info( 2025-05-17 22:08:40,678 - __main__ - INFO - self._api.repo_info( 2025-05-17 22:08:40,678 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/huggingface_hub/utils/_validators.py", line 114, in _inner_fn 2025-05-17 22:08:40,678 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/huggingface_hub/utils/_validators.py", line 114, in _inner_fn 2025-05-17 22:08:40,678 - sglang - INFO - return fn(*args, **kwargs) 2025-05-17 22:08:40,678 - __main__ - INFO - return fn(*args, **kwargs) 2025-05-17 22:08:40,678 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^ 2025-05-17 22:08:40,678 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^^ 2025-05-17 22:08:40,678 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/huggingface_hub/hf_api.py", line 2816, in repo_info 2025-05-17 22:08:40,678 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/huggingface_hub/hf_api.py", line 2816, in repo_info 2025-05-17 22:08:40,678 - sglang - INFO - return method( 2025-05-17 22:08:40,678 - __main__ - INFO - return method( 2025-05-17 22:08:40,679 - sglang - INFO - ^^^^^^^ 2025-05-17 22:08:40,679 - __main__ - INFO - ^^^^^^^ 2025-05-17 22:08:40,679 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/huggingface_hub/utils/_validators.py", line 114, in _inner_fn 2025-05-17 22:08:40,679 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/huggingface_hub/utils/_validators.py", line 114, in _inner_fn 2025-05-17 22:08:40,679 - sglang - INFO - return fn(*args, **kwargs) 2025-05-17 22:08:40,679 - __main__ - INFO - return fn(*args, **kwargs) 2025-05-17 22:08:40,679 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^ 2025-05-17 22:08:40,679 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^^ 2025-05-17 22:08:40,679 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/huggingface_hub/hf_api.py", line 2600, in model_info 2025-05-17 22:08:40,679 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/huggingface_hub/hf_api.py", line 2600, in model_info 2025-05-17 22:08:40,679 - sglang - INFO - r = get_session().get(path, headers=headers, timeout=timeout, params=params) 2025-05-17 22:08:40,679 - __main__ - INFO - r = get_session().get(path, headers=headers, timeout=timeout, params=params) 2025-05-17 22:08:40,679 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-05-17 22:08:40,679 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-05-17 22:08:40,679 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/requests/sessions.py", line 602, in get 2025-05-17 22:08:40,679 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/requests/sessions.py", line 602, in get 2025-05-17 22:08:40,679 - sglang - INFO - return self.request("GET", url, **kwargs) 2025-05-17 22:08:40,679 - __main__ - INFO - return self.request("GET", url, **kwargs) 2025-05-17 22:08:40,679 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-05-17 22:08:40,679 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-05-17 22:08:40,679 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/requests/sessions.py", line 589, in request 2025-05-17 22:08:40,679 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/requests/sessions.py", line 589, in request 2025-05-17 22:08:40,679 - sglang - INFO - resp = self.send(prep, **send_kwargs) 2025-05-17 22:08:40,679 - __main__ - INFO - resp = self.send(prep, **send_kwargs) 2025-05-17 22:08:40,679 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-05-17 22:08:40,679 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-05-17 22:08:40,680 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/requests/sessions.py", line 703, in send 2025-05-17 22:08:40,680 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/requests/sessions.py", line 703, in send 2025-05-17 22:08:40,680 - sglang - INFO - r = adapter.send(request, **kwargs) 2025-05-17 22:08:40,680 - __main__ - INFO - r = adapter.send(request, **kwargs) 2025-05-17 22:08:40,680 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-05-17 22:08:40,680 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-05-17 22:08:40,680 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/huggingface_hub/utils/_http.py", line 96, in send 2025-05-17 22:08:40,680 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/huggingface_hub/utils/_http.py", line 96, in send 2025-05-17 22:08:40,680 - sglang - INFO - return super().send(request, *args, **kwargs) 2025-05-17 22:08:40,680 - __main__ - INFO - return super().send(request, *args, **kwargs) 2025-05-17 22:08:40,680 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-05-17 22:08:40,680 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-05-17 22:08:40,680 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/requests/adapters.py", line 700, in send 2025-05-17 22:08:40,680 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/requests/adapters.py", line 700, in send 2025-05-17 22:08:40,680 - sglang - INFO - raise ConnectionError(e, request=request) 2025-05-17 22:08:40,680 - __main__ - INFO - raise ConnectionError(e, request=request) 2025-05-17 22:08:40,680 - sglang - INFO - requests.exceptions.ConnectionError: (MaxRetryError("HTTPSConnectionPool(host='huggingface.co', port=443): Max retries exceeded with url: /api/models/allenai/olmOCR-7B-0225-preview (Caused by NewConnectionError(': Failed to establish a new connection: [Errno 101] Network is unreachable'))"), '(Request ID: 1d8994fa-4819-4677-98a0-06dfaeccb18c)') 2025-05-17 22:08:40,680 - __main__ - INFO - requests.exceptions.ConnectionError: (MaxRetryError("HTTPSConnectionPool(host='huggingface.co', port=443): Max retries exceeded with url: /api/models/allenai/olmOCR-7B-0225-preview (Caused by NewConnectionError(': Failed to establish a new connection: [Errno 101] Network is unreachable'))"), '(Request ID: 1d8994fa-4819-4677-98a0-06dfaeccb18c)') 2025-05-17 22:08:40,680 - sglang - INFO - 2025-05-17 22:08:40,680 - __main__ - INFO - 2025-05-17 22:08:40,680 - sglang - INFO - [2025-05-17 22:08:40] Received sigquit from a child proces. It usually means the child failed. 2025-05-17 22:08:40,680 - __main__ - INFO - [2025-05-17 22:08:40] Received sigquit from a child proces. It usually means the child failed. 2025-05-17 22:08:40,856 - __main__ - WARNING - SGLang server task ended 2025-05-17 22:08:40,981 - __main__ - WARNING - Attempt 49: Please wait for sglang server to become ready... 2025-05-17 22:08:42,018 - __main__ - WARNING - Attempt 50: Please wait for sglang server to become ready... 2025-05-17 22:08:43,075 - __main__ - WARNING - Attempt 51: Please wait for sglang server to become ready... 2025-05-17 22:08:44,141 - __main__ - WARNING - Attempt 52: Please wait for sglang server to become ready... 2025-05-17 22:08:45,208 - __main__ - WARNING - Attempt 53: Please wait for sglang server to become ready... 2025-05-17 22:08:46,343 - __main__ - WARNING - Attempt 54: Please wait for sglang server to become ready... 2025-05-17 22:08:47,414 - sglang - INFO - [2025-05-17 22:08:47] server_args=ServerArgs(model_path='allenai/olmOCR-7B-0225-preview', tokenizer_path='allenai/olmOCR-7B-0225-preview', tokenizer_mode='auto', load_format='auto', trust_remote_code=False, dtype='auto', kv_cache_dtype='auto', quantization_param_path=None, quantization=None, context_length=None, device='cuda', served_model_name='allenai/olmOCR-7B-0225-preview', chat_template='qwen2-vl', is_embedding=False, revision=None, skip_tokenizer_init=False, host='127.0.0.1', port=30024, mem_fraction_static=0.8, max_running_requests=None, max_total_tokens=None, chunked_prefill_size=2048, max_prefill_tokens=16384, schedule_policy='lpm', schedule_conservativeness=1.0, cpu_offload_gb=0, prefill_only_one_req=False, tp_size=1, stream_interval=1, stream_output=False, random_seed=803034972, constrained_json_whitespace_pattern=None, watchdog_timeout=300, download_dir=None, base_gpu_id=0, log_level='info', log_level_http='warning', log_requests=False, show_time_cost=False, enable_metrics=False, decode_log_interval=40, api_key=None, file_storage_pth='sglang_storage', enable_cache_report=False, dp_size=1, load_balance_method='round_robin', ep_size=1, dist_init_addr=None, nnodes=1, node_rank=0, json_model_override_args='{}', lora_paths=None, max_loras_per_batch=8, attention_backend='flashinfer', sampling_backend='flashinfer', grammar_backend='outlines', speculative_draft_model_path=None, speculative_algorithm=None, speculative_num_steps=5, speculative_num_draft_tokens=64, speculative_eagle_topk=8, enable_double_sparsity=False, ds_channel_config_path=None, ds_heavy_channel_num=32, ds_heavy_token_num=256, ds_heavy_channel_type='qk', ds_sparse_decode_threshold=4096, disable_radix_cache=False, disable_jump_forward=False, disable_cuda_graph=False, disable_cuda_graph_padding=False, disable_outlines_disk_cache=False, disable_custom_all_reduce=False, disable_mla=False, disable_overlap_schedule=False, enable_mixed_chunk=False, enable_dp_attention=False, enable_ep_moe=False, enable_torch_compile=False, torch_compile_max_bs=32, cuda_graph_max_bs=8, cuda_graph_bs=None, torchao_config='', enable_nan_detection=False, enable_p2p_check=False, triton_attention_reduce_in_fp32=False, triton_attention_num_kv_splits=8, num_continuous_decode_steps=1, delete_ckpt_after_loading=False, enable_memory_saver=False, allow_auto_truncate=False, enable_custom_logit_processor=False, tool_call_parser=None) 2025-05-17 22:08:47,414 - __main__ - INFO - [2025-05-17 22:08:47] server_args=ServerArgs(model_path='allenai/olmOCR-7B-0225-preview', tokenizer_path='allenai/olmOCR-7B-0225-preview', tokenizer_mode='auto', load_format='auto', trust_remote_code=False, dtype='auto', kv_cache_dtype='auto', quantization_param_path=None, quantization=None, context_length=None, device='cuda', served_model_name='allenai/olmOCR-7B-0225-preview', chat_template='qwen2-vl', is_embedding=False, revision=None, skip_tokenizer_init=False, host='127.0.0.1', port=30024, mem_fraction_static=0.8, max_running_requests=None, max_total_tokens=None, chunked_prefill_size=2048, max_prefill_tokens=16384, schedule_policy='lpm', schedule_conservativeness=1.0, cpu_offload_gb=0, prefill_only_one_req=False, tp_size=1, stream_interval=1, stream_output=False, random_seed=803034972, constrained_json_whitespace_pattern=None, watchdog_timeout=300, download_dir=None, base_gpu_id=0, log_level='info', log_level_http='warning', log_requests=False, show_time_cost=False, enable_metrics=False, decode_log_interval=40, api_key=None, file_storage_pth='sglang_storage', enable_cache_report=False, dp_size=1, load_balance_method='round_robin', ep_size=1, dist_init_addr=None, nnodes=1, node_rank=0, json_model_override_args='{}', lora_paths=None, max_loras_per_batch=8, attention_backend='flashinfer', sampling_backend='flashinfer', grammar_backend='outlines', speculative_draft_model_path=None, speculative_algorithm=None, speculative_num_steps=5, speculative_num_draft_tokens=64, speculative_eagle_topk=8, enable_double_sparsity=False, ds_channel_config_path=None, ds_heavy_channel_num=32, ds_heavy_token_num=256, ds_heavy_channel_type='qk', ds_sparse_decode_threshold=4096, disable_radix_cache=False, disable_jump_forward=False, disable_cuda_graph=False, disable_cuda_graph_padding=False, disable_outlines_disk_cache=False, disable_custom_all_reduce=False, disable_mla=False, disable_overlap_schedule=False, enable_mixed_chunk=False, enable_dp_attention=False, enable_ep_moe=False, enable_torch_compile=False, torch_compile_max_bs=32, cuda_graph_max_bs=8, cuda_graph_bs=None, torchao_config='', enable_nan_detection=False, enable_p2p_check=False, triton_attention_reduce_in_fp32=False, triton_attention_num_kv_splits=8, num_continuous_decode_steps=1, delete_ckpt_after_loading=False, enable_memory_saver=False, allow_auto_truncate=False, enable_custom_logit_processor=False, tool_call_parser=None) 2025-05-17 22:08:47,415 - __main__ - WARNING - Attempt 55: Please wait for sglang server to become ready... 2025-05-17 22:08:48,487 - __main__ - WARNING - Attempt 56: Please wait for sglang server to become ready... 2025-05-17 22:08:49,520 - __main__ - WARNING - Attempt 57: Please wait for sglang server to become ready... 2025-05-17 22:08:50,581 - __main__ - WARNING - Attempt 58: Please wait for sglang server to become ready... 2025-05-17 22:08:51,650 - __main__ - WARNING - Attempt 59: Please wait for sglang server to become ready... 2025-05-17 22:08:52,707 - __main__ - WARNING - Attempt 60: Please wait for sglang server to become ready... 2025-05-17 22:08:53,775 - __main__ - WARNING - Attempt 61: Please wait for sglang server to become ready... 2025-05-17 22:08:54,258 - sglang - INFO - [2025-05-17 22:08:54] Use chat template for the OpenAI-compatible API server: qwen2-vl 2025-05-17 22:08:54,258 - __main__ - INFO - [2025-05-17 22:08:54] Use chat template for the OpenAI-compatible API server: qwen2-vl 2025-05-17 22:08:54,852 - __main__ - WARNING - Attempt 62: Please wait for sglang server to become ready... 2025-05-17 22:08:55,919 - __main__ - WARNING - Attempt 63: Please wait for sglang server to become ready... 2025-05-17 22:08:56,988 - __main__ - WARNING - Attempt 64: Please wait for sglang server to become ready... 2025-05-17 22:08:58,051 - __main__ - WARNING - Attempt 65: Please wait for sglang server to become ready... 2025-05-17 22:08:59,104 - __main__ - WARNING - Attempt 66: Please wait for sglang server to become ready... 2025-05-17 22:08:59,768 - sglang - INFO - [2025-05-17 22:08:59 TP0] Overlap scheduler is disabled for multimodal models. 2025-05-17 22:08:59,769 - __main__ - INFO - [2025-05-17 22:08:59 TP0] Overlap scheduler is disabled for multimodal models. 2025-05-17 22:08:59,945 - sglang - INFO - [2025-05-17 22:08:59 TP0] Automatically reduce --mem-fraction-static to 0.760 because this is a multimodal model. 2025-05-17 22:08:59,945 - __main__ - INFO - [2025-05-17 22:08:59 TP0] Automatically reduce --mem-fraction-static to 0.760 because this is a multimodal model. 2025-05-17 22:08:59,945 - sglang - INFO - [2025-05-17 22:08:59 TP0] Automatically turn off --chunked-prefill-size and disable radix cache for qwen2-vl. 2025-05-17 22:08:59,945 - __main__ - INFO - [2025-05-17 22:08:59 TP0] Automatically turn off --chunked-prefill-size and disable radix cache for qwen2-vl. 2025-05-17 22:08:59,945 - sglang - INFO - [2025-05-17 22:08:59 TP0] Init torch distributed begin. 2025-05-17 22:08:59,946 - __main__ - INFO - [2025-05-17 22:08:59 TP0] Init torch distributed begin. 2025-05-17 22:09:00,182 - __main__ - WARNING - Attempt 67: Please wait for sglang server to become ready... 2025-05-17 22:09:01,248 - __main__ - WARNING - Attempt 68: Please wait for sglang server to become ready... 2025-05-17 22:09:02,312 - __main__ - WARNING - Attempt 69: Please wait for sglang server to become ready... 2025-05-17 22:09:03,380 - __main__ - WARNING - Attempt 70: Please wait for sglang server to become ready... 2025-05-17 22:09:04,448 - __main__ - WARNING - Attempt 71: Please wait for sglang server to become ready... 2025-05-17 22:09:05,299 - sglang - INFO - [2025-05-17 22:09:05 TP0] Load weight begin. avail mem=23.33 GB 2025-05-17 22:09:05,299 - __main__ - INFO - [2025-05-17 22:09:05 TP0] Load weight begin. avail mem=23.33 GB 2025-05-17 22:09:05,523 - __main__ - WARNING - Attempt 72: Please wait for sglang server to become ready... 2025-05-17 22:09:05,902 - sglang - INFO - [2025-05-17 22:09:05 TP0] Scheduler hit an exception: Traceback (most recent call last): 2025-05-17 22:09:05,902 - __main__ - INFO - [2025-05-17 22:09:05 TP0] Scheduler hit an exception: Traceback (most recent call last): 2025-05-17 22:09:05,903 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/urllib3/connection.py", line 198, in _new_conn 2025-05-17 22:09:05,903 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/urllib3/connection.py", line 198, in _new_conn 2025-05-17 22:09:05,903 - sglang - INFO - sock = connection.create_connection( 2025-05-17 22:09:05,903 - __main__ - INFO - sock = connection.create_connection( 2025-05-17 22:09:05,903 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-05-17 22:09:05,903 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-05-17 22:09:05,903 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/urllib3/util/connection.py", line 85, in create_connection 2025-05-17 22:09:05,903 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/urllib3/util/connection.py", line 85, in create_connection 2025-05-17 22:09:05,903 - sglang - INFO - raise err 2025-05-17 22:09:05,903 - __main__ - INFO - raise err 2025-05-17 22:09:05,903 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/urllib3/util/connection.py", line 73, in create_connection 2025-05-17 22:09:05,903 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/urllib3/util/connection.py", line 73, in create_connection 2025-05-17 22:09:05,903 - sglang - INFO - sock.connect(sa) 2025-05-17 22:09:05,903 - __main__ - INFO - sock.connect(sa) 2025-05-17 22:09:05,903 - sglang - INFO - OSError: [Errno 101] Network is unreachable 2025-05-17 22:09:05,903 - __main__ - INFO - OSError: [Errno 101] Network is unreachable 2025-05-17 22:09:05,903 - sglang - INFO - 2025-05-17 22:09:05,903 - __main__ - INFO - 2025-05-17 22:09:05,903 - sglang - INFO - The above exception was the direct cause of the following exception: 2025-05-17 22:09:05,903 - __main__ - INFO - The above exception was the direct cause of the following exception: 2025-05-17 22:09:05,904 - sglang - INFO - 2025-05-17 22:09:05,904 - __main__ - INFO - 2025-05-17 22:09:05,904 - sglang - INFO - Traceback (most recent call last): 2025-05-17 22:09:05,904 - __main__ - INFO - Traceback (most recent call last): 2025-05-17 22:09:05,904 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/urllib3/connectionpool.py", line 787, in urlopen 2025-05-17 22:09:05,904 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/urllib3/connectionpool.py", line 787, in urlopen 2025-05-17 22:09:05,904 - sglang - INFO - response = self._make_request( 2025-05-17 22:09:05,904 - __main__ - INFO - response = self._make_request( 2025-05-17 22:09:05,904 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^ 2025-05-17 22:09:05,904 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^^ 2025-05-17 22:09:05,904 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/urllib3/connectionpool.py", line 488, in _make_request 2025-05-17 22:09:05,904 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/urllib3/connectionpool.py", line 488, in _make_request 2025-05-17 22:09:05,904 - sglang - INFO - raise new_e 2025-05-17 22:09:05,904 - __main__ - INFO - raise new_e 2025-05-17 22:09:05,904 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/urllib3/connectionpool.py", line 464, in _make_request 2025-05-17 22:09:05,904 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/urllib3/connectionpool.py", line 464, in _make_request 2025-05-17 22:09:05,904 - sglang - INFO - self._validate_conn(conn) 2025-05-17 22:09:05,904 - __main__ - INFO - self._validate_conn(conn) 2025-05-17 22:09:05,904 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/urllib3/connectionpool.py", line 1093, in _validate_conn 2025-05-17 22:09:05,904 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/urllib3/connectionpool.py", line 1093, in _validate_conn 2025-05-17 22:09:05,904 - sglang - INFO - conn.connect() 2025-05-17 22:09:05,904 - __main__ - INFO - conn.connect() 2025-05-17 22:09:05,905 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/urllib3/connection.py", line 704, in connect 2025-05-17 22:09:05,905 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/urllib3/connection.py", line 704, in connect 2025-05-17 22:09:05,905 - sglang - INFO - self.sock = sock = self._new_conn() 2025-05-17 22:09:05,905 - __main__ - INFO - self.sock = sock = self._new_conn() 2025-05-17 22:09:05,905 - sglang - INFO - ^^^^^^^^^^^^^^^^ 2025-05-17 22:09:05,905 - __main__ - INFO - ^^^^^^^^^^^^^^^^ 2025-05-17 22:09:05,905 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/urllib3/connection.py", line 213, in _new_conn 2025-05-17 22:09:05,905 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/urllib3/connection.py", line 213, in _new_conn 2025-05-17 22:09:05,905 - sglang - INFO - raise NewConnectionError( 2025-05-17 22:09:05,905 - __main__ - INFO - raise NewConnectionError( 2025-05-17 22:09:05,905 - sglang - INFO - urllib3.exceptions.NewConnectionError: : Failed to establish a new connection: [Errno 101] Network is unreachable 2025-05-17 22:09:05,905 - __main__ - INFO - urllib3.exceptions.NewConnectionError: : Failed to establish a new connection: [Errno 101] Network is unreachable 2025-05-17 22:09:05,905 - sglang - INFO - 2025-05-17 22:09:05,905 - __main__ - INFO - 2025-05-17 22:09:05,905 - sglang - INFO - The above exception was the direct cause of the following exception: 2025-05-17 22:09:05,905 - __main__ - INFO - The above exception was the direct cause of the following exception: 2025-05-17 22:09:05,905 - sglang - INFO - 2025-05-17 22:09:05,905 - __main__ - INFO - 2025-05-17 22:09:05,905 - sglang - INFO - Traceback (most recent call last): 2025-05-17 22:09:05,905 - __main__ - INFO - Traceback (most recent call last): 2025-05-17 22:09:05,905 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/requests/adapters.py", line 667, in send 2025-05-17 22:09:05,906 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/requests/adapters.py", line 667, in send 2025-05-17 22:09:05,906 - sglang - INFO - resp = conn.urlopen( 2025-05-17 22:09:05,906 - __main__ - INFO - resp = conn.urlopen( 2025-05-17 22:09:05,906 - sglang - INFO - ^^^^^^^^^^^^^ 2025-05-17 22:09:05,906 - __main__ - INFO - ^^^^^^^^^^^^^ 2025-05-17 22:09:05,906 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/urllib3/connectionpool.py", line 841, in urlopen 2025-05-17 22:09:05,906 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/urllib3/connectionpool.py", line 841, in urlopen 2025-05-17 22:09:05,906 - sglang - INFO - retries = retries.increment( 2025-05-17 22:09:05,906 - __main__ - INFO - retries = retries.increment( 2025-05-17 22:09:05,906 - sglang - INFO - ^^^^^^^^^^^^^^^^^^ 2025-05-17 22:09:05,906 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^ 2025-05-17 22:09:05,906 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/urllib3/util/retry.py", line 519, in increment 2025-05-17 22:09:05,906 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/urllib3/util/retry.py", line 519, in increment 2025-05-17 22:09:05,906 - sglang - INFO - raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] 2025-05-17 22:09:05,906 - __main__ - INFO - raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] 2025-05-17 22:09:05,906 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-05-17 22:09:05,906 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-05-17 22:09:05,906 - sglang - INFO - urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='huggingface.co', port=443): Max retries exceeded with url: /api/models/allenai/olmOCR-7B-0225-preview (Caused by NewConnectionError(': Failed to establish a new connection: [Errno 101] Network is unreachable')) 2025-05-17 22:09:05,906 - __main__ - INFO - urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='huggingface.co', port=443): Max retries exceeded with url: /api/models/allenai/olmOCR-7B-0225-preview (Caused by NewConnectionError(': Failed to establish a new connection: [Errno 101] Network is unreachable')) 2025-05-17 22:09:05,906 - sglang - INFO - 2025-05-17 22:09:05,906 - __main__ - INFO - 2025-05-17 22:09:05,907 - sglang - INFO - During handling of the above exception, another exception occurred: 2025-05-17 22:09:05,907 - __main__ - INFO - During handling of the above exception, another exception occurred: 2025-05-17 22:09:05,907 - sglang - INFO - 2025-05-17 22:09:05,907 - __main__ - INFO - 2025-05-17 22:09:05,907 - sglang - INFO - Traceback (most recent call last): 2025-05-17 22:09:05,907 - __main__ - INFO - Traceback (most recent call last): 2025-05-17 22:09:05,907 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/managers/scheduler.py", line 1773, in run_scheduler_process 2025-05-17 22:09:05,907 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/managers/scheduler.py", line 1773, in run_scheduler_process 2025-05-17 22:09:05,907 - sglang - INFO - scheduler = Scheduler(server_args, port_args, gpu_id, tp_rank, dp_rank) 2025-05-17 22:09:05,907 - __main__ - INFO - scheduler = Scheduler(server_args, port_args, gpu_id, tp_rank, dp_rank) 2025-05-17 22:09:05,907 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-05-17 22:09:05,907 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-05-17 22:09:05,907 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/managers/scheduler.py", line 239, in __init__ 2025-05-17 22:09:05,907 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/managers/scheduler.py", line 239, in __init__ 2025-05-17 22:09:05,907 - sglang - INFO - self.tp_worker = TpWorkerClass( 2025-05-17 22:09:05,907 - __main__ - INFO - self.tp_worker = TpWorkerClass( 2025-05-17 22:09:05,907 - sglang - INFO - ^^^^^^^^^^^^^^ 2025-05-17 22:09:05,907 - __main__ - INFO - ^^^^^^^^^^^^^^ 2025-05-17 22:09:05,907 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/managers/tp_worker.py", line 68, in __init__ 2025-05-17 22:09:05,907 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/managers/tp_worker.py", line 68, in __init__ 2025-05-17 22:09:05,907 - sglang - INFO - self.model_runner = ModelRunner( 2025-05-17 22:09:05,907 - __main__ - INFO - self.model_runner = ModelRunner( 2025-05-17 22:09:05,908 - sglang - INFO - ^^^^^^^^^^^^ 2025-05-17 22:09:05,908 - __main__ - INFO - ^^^^^^^^^^^^ 2025-05-17 22:09:05,908 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/model_executor/model_runner.py", line 185, in __init__ 2025-05-17 22:09:05,908 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/model_executor/model_runner.py", line 185, in __init__ 2025-05-17 22:09:05,908 - sglang - INFO - self.load_model() 2025-05-17 22:09:05,908 - __main__ - INFO - self.load_model() 2025-05-17 22:09:05,908 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/model_executor/model_runner.py", line 306, in load_model 2025-05-17 22:09:05,908 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/model_executor/model_runner.py", line 306, in load_model 2025-05-17 22:09:05,908 - sglang - INFO - self.model = get_model( 2025-05-17 22:09:05,908 - __main__ - INFO - self.model = get_model( 2025-05-17 22:09:05,908 - sglang - INFO - ^^^^^^^^^^ 2025-05-17 22:09:05,908 - __main__ - INFO - ^^^^^^^^^^ 2025-05-17 22:09:05,908 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/model_loader/__init__.py", line 22, in get_model 2025-05-17 22:09:05,908 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/model_loader/__init__.py", line 22, in get_model 2025-05-17 22:09:05,908 - sglang - INFO - return loader.load_model( 2025-05-17 22:09:05,908 - __main__ - INFO - return loader.load_model( 2025-05-17 22:09:05,908 - sglang - INFO - ^^^^^^^^^^^^^^^^^^ 2025-05-17 22:09:05,908 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^ 2025-05-17 22:09:05,908 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/model_loader/loader.py", line 362, in load_model 2025-05-17 22:09:05,908 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/model_loader/loader.py", line 362, in load_model 2025-05-17 22:09:05,908 - sglang - INFO - model.load_weights(self._get_all_weights(model_config, model)) 2025-05-17 22:09:05,908 - __main__ - INFO - model.load_weights(self._get_all_weights(model_config, model)) 2025-05-17 22:09:05,909 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/models/qwen2_vl.py", line 576, in load_weights 2025-05-17 22:09:05,909 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/models/qwen2_vl.py", line 576, in load_weights 2025-05-17 22:09:05,909 - sglang - INFO - for name, loaded_weight in weights: 2025-05-17 22:09:05,909 - __main__ - INFO - for name, loaded_weight in weights: 2025-05-17 22:09:05,909 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/model_loader/loader.py", line 335, in _get_all_weights 2025-05-17 22:09:05,909 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/model_loader/loader.py", line 335, in _get_all_weights 2025-05-17 22:09:05,909 - sglang - INFO - yield from self._get_weights_iterator(primary_weights) 2025-05-17 22:09:05,909 - __main__ - INFO - yield from self._get_weights_iterator(primary_weights) 2025-05-17 22:09:05,909 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-05-17 22:09:05,909 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-05-17 22:09:05,909 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/model_loader/loader.py", line 303, in _get_weights_iterator 2025-05-17 22:09:05,909 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/model_loader/loader.py", line 303, in _get_weights_iterator 2025-05-17 22:09:05,909 - sglang - INFO - hf_folder, hf_weights_files, use_safetensors = self._prepare_weights( 2025-05-17 22:09:05,909 - __main__ - INFO - hf_folder, hf_weights_files, use_safetensors = self._prepare_weights( 2025-05-17 22:09:05,909 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^ 2025-05-17 22:09:05,909 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^^^^^ 2025-05-17 22:09:05,909 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/model_loader/loader.py", line 255, in _prepare_weights 2025-05-17 22:09:05,909 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/model_loader/loader.py", line 255, in _prepare_weights 2025-05-17 22:09:05,909 - sglang - INFO - hf_folder = download_weights_from_hf( 2025-05-17 22:09:05,909 - __main__ - INFO - hf_folder = download_weights_from_hf( 2025-05-17 22:09:05,909 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-05-17 22:09:05,909 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-05-17 22:09:05,910 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/model_loader/weight_utils.py", line 246, in download_weights_from_hf 2025-05-17 22:09:05,910 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/model_loader/weight_utils.py", line 246, in download_weights_from_hf 2025-05-17 22:09:05,910 - sglang - INFO - file_list = fs.ls(model_name_or_path, detail=False, revision=revision) 2025-05-17 22:09:05,910 - __main__ - INFO - file_list = fs.ls(model_name_or_path, detail=False, revision=revision) 2025-05-17 22:09:05,910 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-05-17 22:09:05,910 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-05-17 22:09:05,910 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/huggingface_hub/hf_file_system.py", line 368, in ls 2025-05-17 22:09:05,910 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/huggingface_hub/hf_file_system.py", line 368, in ls 2025-05-17 22:09:05,910 - sglang - INFO - resolved_path = self.resolve_path(path, revision=revision) 2025-05-17 22:09:05,910 - __main__ - INFO - resolved_path = self.resolve_path(path, revision=revision) 2025-05-17 22:09:05,910 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-05-17 22:09:05,910 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-05-17 22:09:05,910 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/huggingface_hub/hf_file_system.py", line 209, in resolve_path 2025-05-17 22:09:05,910 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/huggingface_hub/hf_file_system.py", line 209, in resolve_path 2025-05-17 22:09:05,910 - sglang - INFO - repo_and_revision_exist, err = self._repo_and_revision_exist(repo_type, repo_id, revision) 2025-05-17 22:09:05,910 - __main__ - INFO - repo_and_revision_exist, err = self._repo_and_revision_exist(repo_type, repo_id, revision) 2025-05-17 22:09:05,910 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-05-17 22:09:05,910 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-05-17 22:09:05,910 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/huggingface_hub/hf_file_system.py", line 125, in _repo_and_revision_exist 2025-05-17 22:09:05,910 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/huggingface_hub/hf_file_system.py", line 125, in _repo_and_revision_exist 2025-05-17 22:09:05,910 - sglang - INFO - self._api.repo_info( 2025-05-17 22:09:05,910 - __main__ - INFO - self._api.repo_info( 2025-05-17 22:09:05,911 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/huggingface_hub/utils/_validators.py", line 114, in _inner_fn 2025-05-17 22:09:05,911 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/huggingface_hub/utils/_validators.py", line 114, in _inner_fn 2025-05-17 22:09:05,911 - sglang - INFO - return fn(*args, **kwargs) 2025-05-17 22:09:05,911 - __main__ - INFO - return fn(*args, **kwargs) 2025-05-17 22:09:05,911 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^ 2025-05-17 22:09:05,911 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^^ 2025-05-17 22:09:05,911 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/huggingface_hub/hf_api.py", line 2816, in repo_info 2025-05-17 22:09:05,911 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/huggingface_hub/hf_api.py", line 2816, in repo_info 2025-05-17 22:09:05,911 - sglang - INFO - return method( 2025-05-17 22:09:05,911 - __main__ - INFO - return method( 2025-05-17 22:09:05,911 - sglang - INFO - ^^^^^^^ 2025-05-17 22:09:05,911 - __main__ - INFO - ^^^^^^^ 2025-05-17 22:09:05,911 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/huggingface_hub/utils/_validators.py", line 114, in _inner_fn 2025-05-17 22:09:05,911 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/huggingface_hub/utils/_validators.py", line 114, in _inner_fn 2025-05-17 22:09:05,911 - sglang - INFO - return fn(*args, **kwargs) 2025-05-17 22:09:05,911 - __main__ - INFO - return fn(*args, **kwargs) 2025-05-17 22:09:05,911 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^ 2025-05-17 22:09:05,911 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^^ 2025-05-17 22:09:05,911 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/huggingface_hub/hf_api.py", line 2600, in model_info 2025-05-17 22:09:05,911 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/huggingface_hub/hf_api.py", line 2600, in model_info 2025-05-17 22:09:05,911 - sglang - INFO - r = get_session().get(path, headers=headers, timeout=timeout, params=params) 2025-05-17 22:09:05,912 - __main__ - INFO - r = get_session().get(path, headers=headers, timeout=timeout, params=params) 2025-05-17 22:09:05,912 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-05-17 22:09:05,912 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-05-17 22:09:05,912 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/requests/sessions.py", line 602, in get 2025-05-17 22:09:05,912 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/requests/sessions.py", line 602, in get 2025-05-17 22:09:05,912 - sglang - INFO - return self.request("GET", url, **kwargs) 2025-05-17 22:09:05,912 - __main__ - INFO - return self.request("GET", url, **kwargs) 2025-05-17 22:09:05,912 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-05-17 22:09:05,912 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-05-17 22:09:05,912 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/requests/sessions.py", line 589, in request 2025-05-17 22:09:05,912 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/requests/sessions.py", line 589, in request 2025-05-17 22:09:05,912 - sglang - INFO - resp = self.send(prep, **send_kwargs) 2025-05-17 22:09:05,912 - __main__ - INFO - resp = self.send(prep, **send_kwargs) 2025-05-17 22:09:05,912 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-05-17 22:09:05,912 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-05-17 22:09:05,912 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/requests/sessions.py", line 703, in send 2025-05-17 22:09:05,912 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/requests/sessions.py", line 703, in send 2025-05-17 22:09:05,912 - sglang - INFO - r = adapter.send(request, **kwargs) 2025-05-17 22:09:05,912 - __main__ - INFO - r = adapter.send(request, **kwargs) 2025-05-17 22:09:05,912 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-05-17 22:09:05,912 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-05-17 22:09:05,912 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/huggingface_hub/utils/_http.py", line 96, in send 2025-05-17 22:09:05,912 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/huggingface_hub/utils/_http.py", line 96, in send 2025-05-17 22:09:05,913 - sglang - INFO - return super().send(request, *args, **kwargs) 2025-05-17 22:09:05,913 - __main__ - INFO - return super().send(request, *args, **kwargs) 2025-05-17 22:09:05,913 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-05-17 22:09:05,913 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-05-17 22:09:05,913 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/requests/adapters.py", line 700, in send 2025-05-17 22:09:05,913 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/requests/adapters.py", line 700, in send 2025-05-17 22:09:05,913 - sglang - INFO - raise ConnectionError(e, request=request) 2025-05-17 22:09:05,913 - __main__ - INFO - raise ConnectionError(e, request=request) 2025-05-17 22:09:05,913 - sglang - INFO - requests.exceptions.ConnectionError: (MaxRetryError("HTTPSConnectionPool(host='huggingface.co', port=443): Max retries exceeded with url: /api/models/allenai/olmOCR-7B-0225-preview (Caused by NewConnectionError(': Failed to establish a new connection: [Errno 101] Network is unreachable'))"), '(Request ID: 385785cb-3404-4e73-aefc-9a748405b66f)') 2025-05-17 22:09:05,913 - __main__ - INFO - requests.exceptions.ConnectionError: (MaxRetryError("HTTPSConnectionPool(host='huggingface.co', port=443): Max retries exceeded with url: /api/models/allenai/olmOCR-7B-0225-preview (Caused by NewConnectionError(': Failed to establish a new connection: [Errno 101] Network is unreachable'))"), '(Request ID: 385785cb-3404-4e73-aefc-9a748405b66f)') 2025-05-17 22:09:05,913 - sglang - INFO - 2025-05-17 22:09:05,913 - __main__ - INFO - 2025-05-17 22:09:05,913 - sglang - INFO - [2025-05-17 22:09:05] Received sigquit from a child proces. It usually means the child failed. 2025-05-17 22:09:05,913 - __main__ - INFO - [2025-05-17 22:09:05] Received sigquit from a child proces. It usually means the child failed. 2025-05-17 22:09:06,062 - __main__ - WARNING - SGLang server task ended 2025-05-17 22:09:06,600 - __main__ - WARNING - Attempt 73: Please wait for sglang server to become ready... 2025-05-17 22:09:07,669 - __main__ - WARNING - Attempt 74: Please wait for sglang server to become ready... 2025-05-17 22:09:08,728 - __main__ - WARNING - Attempt 75: Please wait for sglang server to become ready... 2025-05-17 22:09:09,792 - __main__ - WARNING - Attempt 76: Please wait for sglang server to become ready... 2025-05-17 22:09:10,394 - __main__ - INFO - Got cancellation request for SGLang server 2025-05-17 22:09:30,402 - __main__ - INFO - Got --pdfs argument, going to add to the work queue 2025-05-17 22:09:30,402 - __main__ - INFO - Loading file at tests/gnarly_pdfs/horribleocr.pdf as PDF document 2025-05-17 22:09:30,402 - __main__ - INFO - Found 1 total pdf paths to add 2025-05-17 22:09:30,405 - __main__ - INFO - Calculated items_per_group: 500 based on average pages per PDF: 1.00 2025-05-17 22:09:30,623 - __main__ - INFO - Starting pipeline with PID 401355 2025-05-17 22:09:30,623 - __main__ - INFO - Downloading model with hugging face 'allenai/olmOCR-7B-0225-preview' 2025-05-17 22:09:35,724 - __main__ - INFO - No work to do, exiting 2025-05-17 22:10:16,045 - __main__ - INFO - Got --pdfs argument, going to add to the work queue 2025-05-17 22:10:16,045 - __main__ - INFO - Loading file at olmocr_workspace/job_1747491009/input.pdf as PDF document 2025-05-17 22:10:16,045 - __main__ - INFO - Found 1 total pdf paths to add 2025-05-17 22:10:16,048 - __main__ - INFO - Calculated items_per_group: 500 based on average pages per PDF: 1.00 2025-05-17 22:10:16,256 - __main__ - INFO - Starting pipeline with PID 401510 2025-05-17 22:10:16,256 - __main__ - INFO - Downloading model with hugging face 'allenai/olmOCR-7B-0225-preview' 2025-05-17 22:10:21,469 - __main__ - WARNING - Attempt 1: Please wait for sglang server to become ready... 2025-05-17 22:10:22,511 - __main__ - WARNING - Attempt 2: Please wait for sglang server to become ready... 2025-05-17 22:10:23,556 - __main__ - WARNING - Attempt 3: Please wait for sglang server to become ready... 2025-05-17 22:10:24,608 - __main__ - WARNING - Attempt 4: Please wait for sglang server to become ready... 2025-05-17 22:10:25,675 - __main__ - WARNING - Attempt 5: Please wait for sglang server to become ready... 2025-05-17 22:10:26,816 - __main__ - WARNING - Attempt 6: Please wait for sglang server to become ready... 2025-05-17 22:10:27,868 - sglang - INFO - [2025-05-17 22:10:27] server_args=ServerArgs(model_path='allenai/olmOCR-7B-0225-preview', tokenizer_path='allenai/olmOCR-7B-0225-preview', tokenizer_mode='auto', load_format='auto', trust_remote_code=False, dtype='auto', kv_cache_dtype='auto', quantization_param_path=None, quantization=None, context_length=None, device='cuda', served_model_name='allenai/olmOCR-7B-0225-preview', chat_template='qwen2-vl', is_embedding=False, revision=None, skip_tokenizer_init=False, host='127.0.0.1', port=30024, mem_fraction_static=0.8, max_running_requests=None, max_total_tokens=None, chunked_prefill_size=2048, max_prefill_tokens=16384, schedule_policy='lpm', schedule_conservativeness=1.0, cpu_offload_gb=0, prefill_only_one_req=False, tp_size=1, stream_interval=1, stream_output=False, random_seed=907351504, constrained_json_whitespace_pattern=None, watchdog_timeout=300, download_dir=None, base_gpu_id=0, log_level='info', log_level_http='warning', log_requests=False, show_time_cost=False, enable_metrics=False, decode_log_interval=40, api_key=None, file_storage_pth='sglang_storage', enable_cache_report=False, dp_size=1, load_balance_method='round_robin', ep_size=1, dist_init_addr=None, nnodes=1, node_rank=0, json_model_override_args='{}', lora_paths=None, max_loras_per_batch=8, attention_backend='flashinfer', sampling_backend='flashinfer', grammar_backend='outlines', speculative_draft_model_path=None, speculative_algorithm=None, speculative_num_steps=5, speculative_num_draft_tokens=64, speculative_eagle_topk=8, enable_double_sparsity=False, ds_channel_config_path=None, ds_heavy_channel_num=32, ds_heavy_token_num=256, ds_heavy_channel_type='qk', ds_sparse_decode_threshold=4096, disable_radix_cache=False, disable_jump_forward=False, disable_cuda_graph=False, disable_cuda_graph_padding=False, disable_outlines_disk_cache=False, disable_custom_all_reduce=False, disable_mla=False, disable_overlap_schedule=False, enable_mixed_chunk=False, enable_dp_attention=False, enable_ep_moe=False, enable_torch_compile=False, torch_compile_max_bs=32, cuda_graph_max_bs=8, cuda_graph_bs=None, torchao_config='', enable_nan_detection=False, enable_p2p_check=False, triton_attention_reduce_in_fp32=False, triton_attention_num_kv_splits=8, num_continuous_decode_steps=1, delete_ckpt_after_loading=False, enable_memory_saver=False, allow_auto_truncate=False, enable_custom_logit_processor=False, tool_call_parser=None) 2025-05-17 22:10:27,868 - __main__ - INFO - [2025-05-17 22:10:27] server_args=ServerArgs(model_path='allenai/olmOCR-7B-0225-preview', tokenizer_path='allenai/olmOCR-7B-0225-preview', tokenizer_mode='auto', load_format='auto', trust_remote_code=False, dtype='auto', kv_cache_dtype='auto', quantization_param_path=None, quantization=None, context_length=None, device='cuda', served_model_name='allenai/olmOCR-7B-0225-preview', chat_template='qwen2-vl', is_embedding=False, revision=None, skip_tokenizer_init=False, host='127.0.0.1', port=30024, mem_fraction_static=0.8, max_running_requests=None, max_total_tokens=None, chunked_prefill_size=2048, max_prefill_tokens=16384, schedule_policy='lpm', schedule_conservativeness=1.0, cpu_offload_gb=0, prefill_only_one_req=False, tp_size=1, stream_interval=1, stream_output=False, random_seed=907351504, constrained_json_whitespace_pattern=None, watchdog_timeout=300, download_dir=None, base_gpu_id=0, log_level='info', log_level_http='warning', log_requests=False, show_time_cost=False, enable_metrics=False, decode_log_interval=40, api_key=None, file_storage_pth='sglang_storage', enable_cache_report=False, dp_size=1, load_balance_method='round_robin', ep_size=1, dist_init_addr=None, nnodes=1, node_rank=0, json_model_override_args='{}', lora_paths=None, max_loras_per_batch=8, attention_backend='flashinfer', sampling_backend='flashinfer', grammar_backend='outlines', speculative_draft_model_path=None, speculative_algorithm=None, speculative_num_steps=5, speculative_num_draft_tokens=64, speculative_eagle_topk=8, enable_double_sparsity=False, ds_channel_config_path=None, ds_heavy_channel_num=32, ds_heavy_token_num=256, ds_heavy_channel_type='qk', ds_sparse_decode_threshold=4096, disable_radix_cache=False, disable_jump_forward=False, disable_cuda_graph=False, disable_cuda_graph_padding=False, disable_outlines_disk_cache=False, disable_custom_all_reduce=False, disable_mla=False, disable_overlap_schedule=False, enable_mixed_chunk=False, enable_dp_attention=False, enable_ep_moe=False, enable_torch_compile=False, torch_compile_max_bs=32, cuda_graph_max_bs=8, cuda_graph_bs=None, torchao_config='', enable_nan_detection=False, enable_p2p_check=False, triton_attention_reduce_in_fp32=False, triton_attention_num_kv_splits=8, num_continuous_decode_steps=1, delete_ckpt_after_loading=False, enable_memory_saver=False, allow_auto_truncate=False, enable_custom_logit_processor=False, tool_call_parser=None) 2025-05-17 22:10:27,869 - __main__ - WARNING - Attempt 7: Please wait for sglang server to become ready... 2025-05-17 22:10:28,903 - __main__ - WARNING - Attempt 8: Please wait for sglang server to become ready... 2025-05-17 22:10:29,963 - __main__ - WARNING - Attempt 9: Please wait for sglang server to become ready... 2025-05-17 22:10:31,030 - __main__ - WARNING - Attempt 10: Please wait for sglang server to become ready... 2025-05-17 22:10:32,098 - __main__ - WARNING - Attempt 11: Please wait for sglang server to become ready... 2025-05-17 22:10:33,166 - __main__ - WARNING - Attempt 12: Please wait for sglang server to become ready... 2025-05-17 22:10:34,232 - __main__ - WARNING - Attempt 13: Please wait for sglang server to become ready... 2025-05-17 22:10:35,266 - __main__ - WARNING - Attempt 14: Please wait for sglang server to become ready... 2025-05-17 22:10:36,326 - __main__ - WARNING - Attempt 15: Please wait for sglang server to become ready... 2025-05-17 22:10:37,398 - __main__ - WARNING - Attempt 16: Please wait for sglang server to become ready... 2025-05-17 22:10:38,466 - __main__ - WARNING - Attempt 17: Please wait for sglang server to become ready... 2025-05-17 22:10:39,529 - __main__ - WARNING - Attempt 18: Please wait for sglang server to become ready... 2025-05-17 22:10:40,585 - __main__ - WARNING - Attempt 19: Please wait for sglang server to become ready... 2025-05-17 22:10:41,651 - __main__ - WARNING - Attempt 20: Please wait for sglang server to become ready... 2025-05-17 22:10:42,718 - __main__ - WARNING - Attempt 21: Please wait for sglang server to become ready... 2025-05-17 22:10:43,747 - __main__ - WARNING - Attempt 22: Please wait for sglang server to become ready... 2025-05-17 22:10:44,782 - __main__ - WARNING - Attempt 23: Please wait for sglang server to become ready... 2025-05-17 22:10:45,845 - __main__ - WARNING - Attempt 24: Please wait for sglang server to become ready... 2025-05-17 22:10:46,913 - __main__ - WARNING - Attempt 25: Please wait for sglang server to become ready... 2025-05-17 22:10:47,981 - __main__ - WARNING - Attempt 26: Please wait for sglang server to become ready... 2025-05-17 22:10:49,050 - __main__ - WARNING - Attempt 27: Please wait for sglang server to become ready... 2025-05-17 22:10:50,118 - __main__ - WARNING - Attempt 28: Please wait for sglang server to become ready... 2025-05-17 22:10:51,187 - __main__ - WARNING - Attempt 29: Please wait for sglang server to become ready... 2025-05-17 22:10:52,255 - __main__ - WARNING - Attempt 30: Please wait for sglang server to become ready... 2025-05-17 22:10:53,323 - __main__ - WARNING - Attempt 31: Please wait for sglang server to become ready... 2025-05-17 22:10:54,391 - __main__ - WARNING - Attempt 32: Please wait for sglang server to become ready... 2025-05-17 22:10:55,459 - __main__ - WARNING - Attempt 33: Please wait for sglang server to become ready... 2025-05-17 22:10:56,527 - __main__ - WARNING - Attempt 34: Please wait for sglang server to become ready... 2025-05-17 22:10:57,590 - __main__ - WARNING - Attempt 35: Please wait for sglang server to become ready... 2025-05-17 22:10:58,647 - __main__ - WARNING - Attempt 36: Please wait for sglang server to become ready... 2025-05-17 22:10:59,714 - __main__ - WARNING - Attempt 37: Please wait for sglang server to become ready... 2025-05-17 22:11:00,781 - __main__ - WARNING - Attempt 38: Please wait for sglang server to become ready... 2025-05-17 22:11:01,848 - __main__ - WARNING - Attempt 39: Please wait for sglang server to become ready... 2025-05-17 22:11:02,919 - __main__ - WARNING - Attempt 40: Please wait for sglang server to become ready... 2025-05-17 22:11:03,954 - __main__ - WARNING - Attempt 41: Please wait for sglang server to become ready... 2025-05-17 22:11:05,021 - __main__ - WARNING - Attempt 42: Please wait for sglang server to become ready... 2025-05-17 22:11:06,088 - __main__ - WARNING - Attempt 43: Please wait for sglang server to become ready... 2025-05-17 22:11:07,156 - __main__ - WARNING - Attempt 44: Please wait for sglang server to become ready... 2025-05-17 22:11:08,223 - __main__ - WARNING - Attempt 45: Please wait for sglang server to become ready... 2025-05-17 22:11:09,291 - __main__ - WARNING - Attempt 46: Please wait for sglang server to become ready... 2025-05-17 22:11:10,363 - __main__ - WARNING - Attempt 47: Please wait for sglang server to become ready... 2025-05-17 22:11:11,419 - __main__ - WARNING - Attempt 48: Please wait for sglang server to become ready... 2025-05-17 22:11:12,487 - __main__ - WARNING - Attempt 49: Please wait for sglang server to become ready... 2025-05-17 22:11:13,555 - __main__ - WARNING - Attempt 50: Please wait for sglang server to become ready... 2025-05-17 22:11:14,628 - __main__ - WARNING - Attempt 51: Please wait for sglang server to become ready... 2025-05-17 22:11:15,684 - __main__ - WARNING - Attempt 52: Please wait for sglang server to become ready... 2025-05-17 22:11:16,747 - __main__ - WARNING - Attempt 53: Please wait for sglang server to become ready... 2025-05-17 22:11:17,814 - __main__ - WARNING - Attempt 54: Please wait for sglang server to become ready... 2025-05-17 22:11:18,880 - __main__ - WARNING - Attempt 55: Please wait for sglang server to become ready... 2025-05-17 22:11:19,948 - __main__ - WARNING - Attempt 56: Please wait for sglang server to become ready... 2025-05-17 22:11:21,015 - __main__ - WARNING - Attempt 57: Please wait for sglang server to become ready... 2025-05-17 22:11:22,091 - __main__ - WARNING - Attempt 58: Please wait for sglang server to become ready... 2025-05-17 22:11:23,155 - __main__ - WARNING - Attempt 59: Please wait for sglang server to become ready... 2025-05-17 22:11:24,228 - __main__ - WARNING - Attempt 60: Please wait for sglang server to become ready... 2025-05-17 22:11:25,295 - __main__ - WARNING - Attempt 61: Please wait for sglang server to become ready... 2025-05-17 22:11:26,362 - __main__ - WARNING - Attempt 62: Please wait for sglang server to become ready... 2025-05-17 22:11:27,430 - __main__ - WARNING - Attempt 63: Please wait for sglang server to become ready... 2025-05-17 22:11:28,498 - __main__ - WARNING - Attempt 64: Please wait for sglang server to become ready... 2025-05-17 22:11:29,562 - __main__ - WARNING - Attempt 65: Please wait for sglang server to become ready... 2025-05-17 22:11:30,629 - __main__ - WARNING - Attempt 66: Please wait for sglang server to become ready... 2025-05-17 22:11:31,693 - __main__ - WARNING - Attempt 67: Please wait for sglang server to become ready... 2025-05-17 22:11:32,745 - __main__ - WARNING - Attempt 68: Please wait for sglang server to become ready... 2025-05-17 22:11:33,811 - __main__ - WARNING - Attempt 69: Please wait for sglang server to become ready... 2025-05-17 22:11:34,880 - __main__ - WARNING - Attempt 70: Please wait for sglang server to become ready... 2025-05-17 22:11:35,947 - __main__ - WARNING - Attempt 71: Please wait for sglang server to become ready... 2025-05-17 22:11:37,014 - __main__ - WARNING - Attempt 72: Please wait for sglang server to become ready... 2025-05-17 22:11:38,082 - __main__ - WARNING - Attempt 73: Please wait for sglang server to become ready... 2025-05-17 22:11:39,150 - __main__ - WARNING - Attempt 74: Please wait for sglang server to become ready... 2025-05-17 22:11:40,218 - __main__ - WARNING - Attempt 75: Please wait for sglang server to become ready... 2025-05-17 22:11:41,287 - __main__ - WARNING - Attempt 76: Please wait for sglang server to become ready... 2025-05-17 22:11:42,354 - __main__ - WARNING - Attempt 77: Please wait for sglang server to become ready... 2025-05-17 22:11:43,423 - __main__ - WARNING - Attempt 78: Please wait for sglang server to become ready... 2025-05-17 22:11:44,495 - __main__ - WARNING - Attempt 79: Please wait for sglang server to become ready... 2025-05-17 22:11:45,567 - __main__ - WARNING - Attempt 80: Please wait for sglang server to become ready... 2025-05-17 22:11:46,639 - __main__ - WARNING - Attempt 81: Please wait for sglang server to become ready... 2025-05-17 22:11:47,707 - __main__ - WARNING - Attempt 82: Please wait for sglang server to become ready... 2025-05-17 22:11:48,771 - __main__ - WARNING - Attempt 83: Please wait for sglang server to become ready... 2025-05-17 22:11:49,826 - __main__ - WARNING - Attempt 84: Please wait for sglang server to become ready... 2025-05-17 22:11:50,893 - __main__ - WARNING - Attempt 85: Please wait for sglang server to become ready... 2025-05-17 22:11:51,962 - __main__ - WARNING - Attempt 86: Please wait for sglang server to become ready... 2025-05-17 22:11:53,028 - __main__ - WARNING - Attempt 87: Please wait for sglang server to become ready... 2025-05-17 22:11:54,096 - __main__ - WARNING - Attempt 88: Please wait for sglang server to become ready... 2025-05-17 22:11:55,163 - __main__ - WARNING - Attempt 89: Please wait for sglang server to become ready... 2025-05-17 22:11:56,231 - __main__ - WARNING - Attempt 90: Please wait for sglang server to become ready... 2025-05-17 22:11:57,304 - __main__ - WARNING - Attempt 91: Please wait for sglang server to become ready... 2025-05-17 22:11:58,372 - __main__ - WARNING - Attempt 92: Please wait for sglang server to become ready... 2025-05-17 22:11:59,441 - __main__ - WARNING - Attempt 93: Please wait for sglang server to become ready... 2025-05-17 22:12:00,509 - __main__ - WARNING - Attempt 94: Please wait for sglang server to become ready... 2025-05-17 22:12:01,577 - __main__ - WARNING - Attempt 95: Please wait for sglang server to become ready... 2025-05-17 22:12:02,645 - __main__ - WARNING - Attempt 96: Please wait for sglang server to become ready... 2025-05-17 22:12:03,713 - __main__ - WARNING - Attempt 97: Please wait for sglang server to become ready... 2025-05-17 22:12:04,323 - sglang - INFO - [2025-05-17 22:12:04] Use chat template for the OpenAI-compatible API server: qwen2-vl 2025-05-17 22:12:04,323 - __main__ - INFO - [2025-05-17 22:12:04] Use chat template for the OpenAI-compatible API server: qwen2-vl 2025-05-17 22:12:04,790 - __main__ - WARNING - Attempt 98: Please wait for sglang server to become ready... 2025-05-17 22:12:05,857 - __main__ - WARNING - Attempt 99: Please wait for sglang server to become ready... 2025-05-17 22:12:06,909 - __main__ - WARNING - Attempt 100: Please wait for sglang server to become ready... 2025-05-17 22:12:07,976 - __main__ - WARNING - Attempt 101: Please wait for sglang server to become ready... 2025-05-17 22:12:09,043 - __main__ - WARNING - Attempt 102: Please wait for sglang server to become ready... 2025-05-17 22:12:10,111 - __main__ - WARNING - Attempt 103: Please wait for sglang server to become ready... 2025-05-17 22:12:10,225 - sglang - INFO - [2025-05-17 22:12:10 TP0] Overlap scheduler is disabled for multimodal models. 2025-05-17 22:12:10,225 - __main__ - INFO - [2025-05-17 22:12:10 TP0] Overlap scheduler is disabled for multimodal models. 2025-05-17 22:12:11,188 - __main__ - WARNING - Attempt 104: Please wait for sglang server to become ready... 2025-05-17 22:12:12,256 - __main__ - WARNING - Attempt 105: Please wait for sglang server to become ready... 2025-05-17 22:12:13,325 - __main__ - WARNING - Attempt 106: Please wait for sglang server to become ready... 2025-05-17 22:12:14,393 - __main__ - WARNING - Attempt 107: Please wait for sglang server to become ready... 2025-05-17 22:12:15,462 - __main__ - WARNING - Attempt 108: Please wait for sglang server to become ready... 2025-05-17 22:12:16,531 - __main__ - WARNING - Attempt 109: Please wait for sglang server to become ready... 2025-05-17 22:12:17,604 - __main__ - WARNING - Attempt 110: Please wait for sglang server to become ready... 2025-05-17 22:12:18,672 - __main__ - WARNING - Attempt 111: Please wait for sglang server to become ready... 2025-05-17 22:12:19,740 - __main__ - WARNING - Attempt 112: Please wait for sglang server to become ready... 2025-05-17 22:12:20,809 - __main__ - WARNING - Attempt 113: Please wait for sglang server to become ready... 2025-05-17 22:12:21,875 - __main__ - WARNING - Attempt 114: Please wait for sglang server to become ready... 2025-05-17 22:12:22,936 - __main__ - WARNING - Attempt 115: Please wait for sglang server to become ready... 2025-05-17 22:12:23,989 - __main__ - WARNING - Attempt 116: Please wait for sglang server to become ready... 2025-05-17 22:12:25,051 - __main__ - WARNING - Attempt 117: Please wait for sglang server to become ready... 2025-05-17 22:12:26,120 - __main__ - WARNING - Attempt 118: Please wait for sglang server to become ready... 2025-05-17 22:12:27,188 - __main__ - WARNING - Attempt 119: Please wait for sglang server to become ready... 2025-05-17 22:12:28,152 - __main__ - INFO - Got cancellation request for SGLang server 2025-05-17 22:13:07,270 - __main__ - INFO - Got --pdfs argument, going to add to the work queue 2025-05-17 22:13:07,271 - __main__ - INFO - Loading file at olmocr_workspace/job_1747491180/input.pdf as PDF document 2025-05-17 22:13:07,271 - __main__ - INFO - Found 1 total pdf paths to add 2025-05-17 22:13:07,273 - __main__ - INFO - Calculated items_per_group: 500 based on average pages per PDF: 1.00 2025-05-17 22:13:07,567 - __main__ - INFO - Starting pipeline with PID 402318 2025-05-17 22:13:07,567 - __main__ - INFO - Downloading model with hugging face 'allenai/olmOCR-7B-0225-preview' 2025-05-17 22:15:43,322 - __main__ - INFO - Got --pdfs argument, going to add to the work queue 2025-05-17 22:15:43,322 - __main__ - INFO - Loading file at olmocr_workspace/job_1747491337/input.pdf as PDF document 2025-05-17 22:15:43,322 - __main__ - INFO - Found 1 total pdf paths to add 2025-05-17 22:15:43,324 - __main__ - INFO - Calculated items_per_group: 500 based on average pages per PDF: 1.00 2025-05-17 22:15:43,625 - __main__ - INFO - Starting pipeline with PID 402524 2025-05-17 22:15:43,625 - __main__ - INFO - Downloading model with hugging face 'allenai/olmOCR-7B-0225-preview' 2025-05-17 22:17:59,962 - __main__ - WARNING - Attempt 1: Please wait for sglang server to become ready... 2025-05-17 22:18:01,003 - __main__ - WARNING - Attempt 2: Please wait for sglang server to become ready... 2025-05-17 22:18:01,262 - __main__ - INFO - Got cancellation request for SGLang server 2025-05-17 22:21:02,917 - __main__ - INFO - Got --pdfs argument, going to add to the work queue 2025-05-17 22:21:02,918 - __main__ - INFO - Loading file at tests/gnarly_pdfs/horribleocr.pdf as PDF document 2025-05-17 22:21:02,918 - __main__ - INFO - Found 1 total pdf paths to add 2025-05-17 22:21:02,920 - __main__ - INFO - Calculated items_per_group: 500 based on average pages per PDF: 1.00 2025-05-17 22:21:03,172 - __main__ - INFO - Starting pipeline with PID 404165 2025-05-17 22:21:03,172 - __main__ - INFO - Downloading model with hugging face 'allenai/olmOCR-7B-0225-preview' 2025-05-17 22:23:19,332 - __main__ - INFO - No work to do, exiting 2025-05-17 22:27:55,694 - __main__ - INFO - Got --pdfs argument, going to add to the work queue 2025-05-17 22:27:55,694 - __main__ - INFO - Loading file at olmocr_workspace/job_1747492069/input.pdf as PDF document 2025-05-17 22:27:55,694 - __main__ - INFO - Found 1 total pdf paths to add 2025-05-17 22:27:55,696 - __main__ - INFO - Calculated items_per_group: 500 based on average pages per PDF: 1.00 2025-05-17 22:27:55,922 - __main__ - INFO - Starting pipeline with PID 404544 2025-05-17 22:27:55,922 - __main__ - INFO - Downloading model with hugging face 'allenai/olmOCR-7B-0225-preview' 2025-05-17 22:30:11,099 - __main__ - WARNING - Attempt 1: Please wait for sglang server to become ready... 2025-05-17 22:30:12,145 - __main__ - WARNING - Attempt 2: Please wait for sglang server to become ready... 2025-05-17 22:30:13,201 - __main__ - WARNING - Attempt 3: Please wait for sglang server to become ready... 2025-05-17 22:30:14,262 - __main__ - WARNING - Attempt 4: Please wait for sglang server to become ready... 2025-05-17 22:30:15,329 - __main__ - WARNING - Attempt 5: Please wait for sglang server to become ready... 2025-05-17 22:30:16,395 - __main__ - WARNING - Attempt 6: Please wait for sglang server to become ready... 2025-05-17 22:30:17,281 - sglang - INFO - [2025-05-17 22:30:17] server_args=ServerArgs(model_path='allenai/olmOCR-7B-0225-preview', tokenizer_path='allenai/olmOCR-7B-0225-preview', tokenizer_mode='auto', load_format='auto', trust_remote_code=False, dtype='auto', kv_cache_dtype='auto', quantization_param_path=None, quantization=None, context_length=None, device='cuda', served_model_name='allenai/olmOCR-7B-0225-preview', chat_template='qwen2-vl', is_embedding=False, revision=None, skip_tokenizer_init=False, host='127.0.0.1', port=30024, mem_fraction_static=0.8, max_running_requests=None, max_total_tokens=None, chunked_prefill_size=2048, max_prefill_tokens=16384, schedule_policy='lpm', schedule_conservativeness=1.0, cpu_offload_gb=0, prefill_only_one_req=False, tp_size=1, stream_interval=1, stream_output=False, random_seed=403061725, constrained_json_whitespace_pattern=None, watchdog_timeout=300, download_dir=None, base_gpu_id=0, log_level='info', log_level_http='warning', log_requests=False, show_time_cost=False, enable_metrics=False, decode_log_interval=40, api_key=None, file_storage_pth='sglang_storage', enable_cache_report=False, dp_size=1, load_balance_method='round_robin', ep_size=1, dist_init_addr=None, nnodes=1, node_rank=0, json_model_override_args='{}', lora_paths=None, max_loras_per_batch=8, attention_backend='flashinfer', sampling_backend='flashinfer', grammar_backend='outlines', speculative_draft_model_path=None, speculative_algorithm=None, speculative_num_steps=5, speculative_num_draft_tokens=64, speculative_eagle_topk=8, enable_double_sparsity=False, ds_channel_config_path=None, ds_heavy_channel_num=32, ds_heavy_token_num=256, ds_heavy_channel_type='qk', ds_sparse_decode_threshold=4096, disable_radix_cache=False, disable_jump_forward=False, disable_cuda_graph=False, disable_cuda_graph_padding=False, disable_outlines_disk_cache=False, disable_custom_all_reduce=False, disable_mla=False, disable_overlap_schedule=False, enable_mixed_chunk=False, enable_dp_attention=False, enable_ep_moe=False, enable_torch_compile=False, torch_compile_max_bs=32, cuda_graph_max_bs=8, cuda_graph_bs=None, torchao_config='', enable_nan_detection=False, enable_p2p_check=False, triton_attention_reduce_in_fp32=False, triton_attention_num_kv_splits=8, num_continuous_decode_steps=1, delete_ckpt_after_loading=False, enable_memory_saver=False, allow_auto_truncate=False, enable_custom_logit_processor=False, tool_call_parser=None) 2025-05-17 22:30:17,281 - __main__ - INFO - [2025-05-17 22:30:17] server_args=ServerArgs(model_path='allenai/olmOCR-7B-0225-preview', tokenizer_path='allenai/olmOCR-7B-0225-preview', tokenizer_mode='auto', load_format='auto', trust_remote_code=False, dtype='auto', kv_cache_dtype='auto', quantization_param_path=None, quantization=None, context_length=None, device='cuda', served_model_name='allenai/olmOCR-7B-0225-preview', chat_template='qwen2-vl', is_embedding=False, revision=None, skip_tokenizer_init=False, host='127.0.0.1', port=30024, mem_fraction_static=0.8, max_running_requests=None, max_total_tokens=None, chunked_prefill_size=2048, max_prefill_tokens=16384, schedule_policy='lpm', schedule_conservativeness=1.0, cpu_offload_gb=0, prefill_only_one_req=False, tp_size=1, stream_interval=1, stream_output=False, random_seed=403061725, constrained_json_whitespace_pattern=None, watchdog_timeout=300, download_dir=None, base_gpu_id=0, log_level='info', log_level_http='warning', log_requests=False, show_time_cost=False, enable_metrics=False, decode_log_interval=40, api_key=None, file_storage_pth='sglang_storage', enable_cache_report=False, dp_size=1, load_balance_method='round_robin', ep_size=1, dist_init_addr=None, nnodes=1, node_rank=0, json_model_override_args='{}', lora_paths=None, max_loras_per_batch=8, attention_backend='flashinfer', sampling_backend='flashinfer', grammar_backend='outlines', speculative_draft_model_path=None, speculative_algorithm=None, speculative_num_steps=5, speculative_num_draft_tokens=64, speculative_eagle_topk=8, enable_double_sparsity=False, ds_channel_config_path=None, ds_heavy_channel_num=32, ds_heavy_token_num=256, ds_heavy_channel_type='qk', ds_sparse_decode_threshold=4096, disable_radix_cache=False, disable_jump_forward=False, disable_cuda_graph=False, disable_cuda_graph_padding=False, disable_outlines_disk_cache=False, disable_custom_all_reduce=False, disable_mla=False, disable_overlap_schedule=False, enable_mixed_chunk=False, enable_dp_attention=False, enable_ep_moe=False, enable_torch_compile=False, torch_compile_max_bs=32, cuda_graph_max_bs=8, cuda_graph_bs=None, torchao_config='', enable_nan_detection=False, enable_p2p_check=False, triton_attention_reduce_in_fp32=False, triton_attention_num_kv_splits=8, num_continuous_decode_steps=1, delete_ckpt_after_loading=False, enable_memory_saver=False, allow_auto_truncate=False, enable_custom_logit_processor=False, tool_call_parser=None) 2025-05-17 22:30:17,553 - __main__ - WARNING - Attempt 7: Please wait for sglang server to become ready... 2025-05-17 22:30:18,621 - __main__ - WARNING - Attempt 8: Please wait for sglang server to become ready... 2025-05-17 22:30:19,665 - __main__ - WARNING - Attempt 9: Please wait for sglang server to become ready... 2025-05-17 22:30:20,711 - __main__ - WARNING - Attempt 10: Please wait for sglang server to become ready... 2025-05-17 22:30:21,758 - __main__ - WARNING - Attempt 11: Please wait for sglang server to become ready... 2025-05-17 22:30:22,798 - __main__ - WARNING - Attempt 12: Please wait for sglang server to become ready... 2025-05-17 22:30:23,842 - __main__ - WARNING - Attempt 13: Please wait for sglang server to become ready... 2025-05-17 22:30:24,878 - __main__ - WARNING - Attempt 14: Please wait for sglang server to become ready... 2025-05-17 22:30:25,941 - __main__ - WARNING - Attempt 15: Please wait for sglang server to become ready... 2025-05-17 22:30:27,008 - __main__ - WARNING - Attempt 16: Please wait for sglang server to become ready... 2025-05-17 22:30:28,077 - __main__ - WARNING - Attempt 17: Please wait for sglang server to become ready... 2025-05-17 22:30:29,145 - __main__ - WARNING - Attempt 18: Please wait for sglang server to become ready... 2025-05-17 22:30:30,209 - __main__ - WARNING - Attempt 19: Please wait for sglang server to become ready... 2025-05-17 22:30:31,261 - __main__ - WARNING - Attempt 20: Please wait for sglang server to become ready... 2025-05-17 22:30:32,328 - __main__ - WARNING - Attempt 21: Please wait for sglang server to become ready... 2025-05-17 22:30:33,391 - __main__ - WARNING - Attempt 22: Please wait for sglang server to become ready... 2025-05-17 22:30:34,459 - __main__ - WARNING - Attempt 23: Please wait for sglang server to become ready... 2025-05-17 22:30:35,526 - __main__ - WARNING - Attempt 24: Please wait for sglang server to become ready... 2025-05-17 22:30:36,595 - __main__ - WARNING - Attempt 25: Please wait for sglang server to become ready... 2025-05-17 22:30:37,663 - __main__ - WARNING - Attempt 26: Please wait for sglang server to become ready... 2025-05-17 22:30:38,736 - __main__ - WARNING - Attempt 27: Please wait for sglang server to become ready... 2025-05-17 22:30:39,803 - __main__ - WARNING - Attempt 28: Please wait for sglang server to become ready... 2025-05-17 22:30:40,873 - __main__ - WARNING - Attempt 29: Please wait for sglang server to become ready... 2025-05-17 22:30:41,941 - __main__ - WARNING - Attempt 30: Please wait for sglang server to become ready... 2025-05-17 22:30:43,011 - __main__ - WARNING - Attempt 31: Please wait for sglang server to become ready... 2025-05-17 22:30:44,084 - __main__ - WARNING - Attempt 32: Please wait for sglang server to become ready... 2025-05-17 22:30:45,152 - __main__ - WARNING - Attempt 33: Please wait for sglang server to become ready... 2025-05-17 22:30:46,217 - __main__ - WARNING - Attempt 34: Please wait for sglang server to become ready... 2025-05-17 22:30:47,277 - __main__ - WARNING - Attempt 35: Please wait for sglang server to become ready... 2025-05-17 22:30:48,330 - __main__ - WARNING - Attempt 36: Please wait for sglang server to become ready... 2025-05-17 22:30:49,393 - __main__ - WARNING - Attempt 37: Please wait for sglang server to become ready... 2025-05-17 22:30:50,460 - __main__ - WARNING - Attempt 38: Please wait for sglang server to become ready... 2025-05-17 22:30:51,527 - __main__ - WARNING - Attempt 39: Please wait for sglang server to become ready... 2025-05-17 22:30:52,600 - __main__ - WARNING - Attempt 40: Please wait for sglang server to become ready... 2025-05-17 22:30:53,668 - __main__ - WARNING - Attempt 41: Please wait for sglang server to become ready... 2025-05-17 22:30:54,735 - __main__ - WARNING - Attempt 42: Please wait for sglang server to become ready... 2025-05-17 22:30:55,804 - __main__ - WARNING - Attempt 43: Please wait for sglang server to become ready... 2025-05-17 22:30:56,872 - __main__ - WARNING - Attempt 44: Please wait for sglang server to become ready... 2025-05-17 22:30:57,940 - __main__ - WARNING - Attempt 45: Please wait for sglang server to become ready... 2025-05-17 22:30:59,004 - __main__ - WARNING - Attempt 46: Please wait for sglang server to become ready... 2025-05-17 22:31:00,072 - __main__ - WARNING - Attempt 47: Please wait for sglang server to become ready... 2025-05-17 22:31:01,140 - __main__ - WARNING - Attempt 48: Please wait for sglang server to become ready... 2025-05-17 22:31:02,208 - __main__ - WARNING - Attempt 49: Please wait for sglang server to become ready... 2025-05-17 22:31:03,275 - __main__ - WARNING - Attempt 50: Please wait for sglang server to become ready... 2025-05-17 22:31:04,339 - __main__ - WARNING - Attempt 51: Please wait for sglang server to become ready... 2025-05-17 22:31:05,392 - __main__ - WARNING - Attempt 52: Please wait for sglang server to become ready... 2025-05-17 22:31:06,455 - __main__ - WARNING - Attempt 53: Please wait for sglang server to become ready... 2025-05-17 22:31:07,527 - __main__ - WARNING - Attempt 54: Please wait for sglang server to become ready... 2025-05-17 22:31:08,595 - __main__ - WARNING - Attempt 55: Please wait for sglang server to become ready... 2025-05-17 22:31:09,667 - __main__ - WARNING - Attempt 56: Please wait for sglang server to become ready... 2025-05-17 22:31:10,735 - __main__ - WARNING - Attempt 57: Please wait for sglang server to become ready... 2025-05-17 22:31:11,808 - __main__ - WARNING - Attempt 58: Please wait for sglang server to become ready... 2025-05-17 22:31:12,879 - __main__ - WARNING - Attempt 59: Please wait for sglang server to become ready... 2025-05-17 22:31:13,952 - __main__ - WARNING - Attempt 60: Please wait for sglang server to become ready... 2025-05-17 22:31:15,020 - __main__ - WARNING - Attempt 61: Please wait for sglang server to become ready... 2025-05-17 22:31:16,088 - __main__ - WARNING - Attempt 62: Please wait for sglang server to become ready... 2025-05-17 22:31:17,156 - __main__ - WARNING - Attempt 63: Please wait for sglang server to become ready... 2025-05-17 22:31:18,224 - __main__ - WARNING - Attempt 64: Please wait for sglang server to become ready... 2025-05-17 22:31:19,292 - __main__ - WARNING - Attempt 65: Please wait for sglang server to become ready... 2025-05-17 22:31:20,356 - __main__ - WARNING - Attempt 66: Please wait for sglang server to become ready... 2025-05-17 22:31:21,416 - __main__ - WARNING - Attempt 67: Please wait for sglang server to become ready... 2025-05-17 22:31:22,469 - __main__ - WARNING - Attempt 68: Please wait for sglang server to become ready... 2025-05-17 22:31:23,532 - __main__ - WARNING - Attempt 69: Please wait for sglang server to become ready... 2025-05-17 22:31:24,599 - __main__ - WARNING - Attempt 70: Please wait for sglang server to become ready... 2025-05-17 22:31:25,671 - __main__ - WARNING - Attempt 71: Please wait for sglang server to become ready... 2025-05-17 22:31:26,739 - __main__ - WARNING - Attempt 72: Please wait for sglang server to become ready... 2025-05-17 22:31:27,807 - __main__ - WARNING - Attempt 73: Please wait for sglang server to become ready... 2025-05-17 22:31:28,876 - __main__ - WARNING - Attempt 74: Please wait for sglang server to become ready... 2025-05-17 22:31:29,945 - __main__ - WARNING - Attempt 75: Please wait for sglang server to become ready... 2025-05-17 22:31:31,013 - __main__ - WARNING - Attempt 76: Please wait for sglang server to become ready... 2025-05-17 22:31:31,749 - __main__ - INFO - Got cancellation request for SGLang server 2025-05-17 22:31:40,102 - __main__ - INFO - Got --pdfs argument, going to add to the work queue 2025-05-17 22:31:40,102 - __main__ - INFO - Loading file at tests/gnarly_pdfs/horribleocr.pdf as PDF document 2025-05-17 22:31:40,102 - __main__ - INFO - Found 1 total pdf paths to add 2025-05-17 22:31:40,106 - __main__ - INFO - Calculated items_per_group: 500 based on average pages per PDF: 1.00 2025-05-17 22:31:40,324 - __main__ - INFO - Starting pipeline with PID 405667 2025-05-17 22:31:40,324 - __main__ - INFO - Downloading model with hugging face 'allenai/olmOCR-7B-0225-preview' 2025-05-17 22:34:59,610 - __main__ - INFO - Got --pdfs argument, going to add to the work queue 2025-05-17 22:34:59,610 - __main__ - INFO - Loading file at tests/gnarly_pdfs/horribleocr.pdf as PDF document 2025-05-17 22:34:59,610 - __main__ - INFO - Found 1 total pdf paths to add 2025-05-17 22:34:59,613 - __main__ - INFO - Calculated items_per_group: 500 based on average pages per PDF: 1.00 2025-05-17 22:34:59,876 - __main__ - INFO - Starting pipeline with PID 405958 2025-05-17 22:34:59,876 - __main__ - INFO - Downloading model with hugging face 'allenai/olmOCR-7B-0225-preview' 2025-05-17 22:37:14,916 - __main__ - INFO - No work to do, exiting 2025-05-17 22:45:58,117 - __main__ - INFO - Got --pdfs argument, going to add to the work queue 2025-05-17 22:45:58,118 - __main__ - INFO - Loading file at tests/gnarly_pdfs/horribleocr.pdf as PDF document 2025-05-17 22:45:58,118 - __main__ - INFO - Found 1 total pdf paths to add 2025-05-17 22:45:58,121 - __main__ - INFO - Calculated items_per_group: 500 based on average pages per PDF: 1.00 2025-05-17 22:45:58,379 - __main__ - INFO - Starting pipeline with PID 406610 2025-05-17 22:45:58,379 - __main__ - INFO - Downloading model with hugging face 'allenai/olmOCR-7B-0225-preview' 2025-05-17 22:48:14,372 - __main__ - INFO - No work to do, exiting 2025-05-17 22:48:42,758 - __main__ - INFO - Got --pdfs argument, going to add to the work queue 2025-05-17 22:48:42,758 - __main__ - INFO - Loading file at tests/gnarly_pdfs/horribleocr.pdf as PDF document 2025-05-17 22:48:42,758 - __main__ - INFO - Found 1 total pdf paths to add 2025-05-17 22:48:42,762 - __main__ - INFO - Calculated items_per_group: 500 based on average pages per PDF: 1.00 2025-05-17 22:48:43,037 - __main__ - INFO - Starting pipeline with PID 407353 2025-05-17 22:48:43,037 - __main__ - INFO - Downloading model with hugging face 'allenai/olmOCR-7B-0225-preview' 2025-05-17 22:51:38,663 - __main__ - INFO - Got --pdfs argument, going to add to the work queue 2025-05-17 22:51:38,663 - __main__ - INFO - Loading file at tests/gnarly_pdfs/horribleocr.pdf as PDF document 2025-05-17 22:51:38,663 - __main__ - INFO - Found 1 total pdf paths to add 2025-05-17 22:51:38,666 - __main__ - INFO - Calculated items_per_group: 500 based on average pages per PDF: 1.00 2025-05-17 22:51:38,887 - __main__ - INFO - Starting pipeline with PID 407920 2025-05-17 22:51:38,887 - __main__ - INFO - Downloading model with hugging face 'allenai/olmOCR-7B-0225-preview' 2025-05-17 22:51:44,965 - __main__ - INFO - No work to do, exiting 2025-05-17 22:52:53,522 - __main__ - INFO - Got --pdfs argument, going to add to the work queue 2025-05-17 22:52:53,522 - __main__ - INFO - Loading file at olmocr_workspace/job_1747493567/input.pdf as PDF document 2025-05-17 22:52:53,522 - __main__ - INFO - Found 1 total pdf paths to add 2025-05-17 22:52:53,524 - __main__ - INFO - Calculated items_per_group: 500 based on average pages per PDF: 1.00 2025-05-17 22:52:53,743 - __main__ - INFO - Starting pipeline with PID 408294 2025-05-17 22:52:53,743 - __main__ - INFO - Downloading model with hugging face 'allenai/olmOCR-7B-0225-preview' 2025-05-17 22:52:59,333 - __main__ - WARNING - Attempt 1: Please wait for sglang server to become ready... 2025-05-17 22:53:00,367 - __main__ - WARNING - Attempt 2: Please wait for sglang server to become ready... 2025-05-17 22:53:01,421 - __main__ - WARNING - Attempt 3: Please wait for sglang server to become ready... 2025-05-17 22:53:02,488 - __main__ - WARNING - Attempt 4: Please wait for sglang server to become ready... 2025-05-17 22:53:03,554 - __main__ - WARNING - Attempt 5: Please wait for sglang server to become ready... 2025-05-17 22:53:04,629 - __main__ - WARNING - Attempt 6: Please wait for sglang server to become ready... 2025-05-17 22:53:05,307 - sglang - INFO - [2025-05-17 22:53:05] server_args=ServerArgs(model_path='allenai/olmOCR-7B-0225-preview', tokenizer_path='allenai/olmOCR-7B-0225-preview', tokenizer_mode='auto', load_format='auto', trust_remote_code=False, dtype='auto', kv_cache_dtype='auto', quantization_param_path=None, quantization=None, context_length=None, device='cuda', served_model_name='allenai/olmOCR-7B-0225-preview', chat_template='qwen2-vl', is_embedding=False, revision=None, skip_tokenizer_init=False, host='127.0.0.1', port=30024, mem_fraction_static=0.8, max_running_requests=None, max_total_tokens=None, chunked_prefill_size=2048, max_prefill_tokens=16384, schedule_policy='lpm', schedule_conservativeness=1.0, cpu_offload_gb=0, prefill_only_one_req=False, tp_size=1, stream_interval=1, stream_output=False, random_seed=6928412, constrained_json_whitespace_pattern=None, watchdog_timeout=300, download_dir=None, base_gpu_id=0, log_level='info', log_level_http='warning', log_requests=False, show_time_cost=False, enable_metrics=False, decode_log_interval=40, api_key=None, file_storage_pth='sglang_storage', enable_cache_report=False, dp_size=1, load_balance_method='round_robin', ep_size=1, dist_init_addr=None, nnodes=1, node_rank=0, json_model_override_args='{}', lora_paths=None, max_loras_per_batch=8, attention_backend='flashinfer', sampling_backend='flashinfer', grammar_backend='outlines', speculative_draft_model_path=None, speculative_algorithm=None, speculative_num_steps=5, speculative_num_draft_tokens=64, speculative_eagle_topk=8, enable_double_sparsity=False, ds_channel_config_path=None, ds_heavy_channel_num=32, ds_heavy_token_num=256, ds_heavy_channel_type='qk', ds_sparse_decode_threshold=4096, disable_radix_cache=False, disable_jump_forward=False, disable_cuda_graph=False, disable_cuda_graph_padding=False, disable_outlines_disk_cache=False, disable_custom_all_reduce=False, disable_mla=False, disable_overlap_schedule=False, enable_mixed_chunk=False, enable_dp_attention=False, enable_ep_moe=False, enable_torch_compile=False, torch_compile_max_bs=32, cuda_graph_max_bs=8, cuda_graph_bs=None, torchao_config='', enable_nan_detection=False, enable_p2p_check=False, triton_attention_reduce_in_fp32=False, triton_attention_num_kv_splits=8, num_continuous_decode_steps=1, delete_ckpt_after_loading=False, enable_memory_saver=False, allow_auto_truncate=False, enable_custom_logit_processor=False, tool_call_parser=None) 2025-05-17 22:53:05,307 - __main__ - INFO - [2025-05-17 22:53:05] server_args=ServerArgs(model_path='allenai/olmOCR-7B-0225-preview', tokenizer_path='allenai/olmOCR-7B-0225-preview', tokenizer_mode='auto', load_format='auto', trust_remote_code=False, dtype='auto', kv_cache_dtype='auto', quantization_param_path=None, quantization=None, context_length=None, device='cuda', served_model_name='allenai/olmOCR-7B-0225-preview', chat_template='qwen2-vl', is_embedding=False, revision=None, skip_tokenizer_init=False, host='127.0.0.1', port=30024, mem_fraction_static=0.8, max_running_requests=None, max_total_tokens=None, chunked_prefill_size=2048, max_prefill_tokens=16384, schedule_policy='lpm', schedule_conservativeness=1.0, cpu_offload_gb=0, prefill_only_one_req=False, tp_size=1, stream_interval=1, stream_output=False, random_seed=6928412, constrained_json_whitespace_pattern=None, watchdog_timeout=300, download_dir=None, base_gpu_id=0, log_level='info', log_level_http='warning', log_requests=False, show_time_cost=False, enable_metrics=False, decode_log_interval=40, api_key=None, file_storage_pth='sglang_storage', enable_cache_report=False, dp_size=1, load_balance_method='round_robin', ep_size=1, dist_init_addr=None, nnodes=1, node_rank=0, json_model_override_args='{}', lora_paths=None, max_loras_per_batch=8, attention_backend='flashinfer', sampling_backend='flashinfer', grammar_backend='outlines', speculative_draft_model_path=None, speculative_algorithm=None, speculative_num_steps=5, speculative_num_draft_tokens=64, speculative_eagle_topk=8, enable_double_sparsity=False, ds_channel_config_path=None, ds_heavy_channel_num=32, ds_heavy_token_num=256, ds_heavy_channel_type='qk', ds_sparse_decode_threshold=4096, disable_radix_cache=False, disable_jump_forward=False, disable_cuda_graph=False, disable_cuda_graph_padding=False, disable_outlines_disk_cache=False, disable_custom_all_reduce=False, disable_mla=False, disable_overlap_schedule=False, enable_mixed_chunk=False, enable_dp_attention=False, enable_ep_moe=False, enable_torch_compile=False, torch_compile_max_bs=32, cuda_graph_max_bs=8, cuda_graph_bs=None, torchao_config='', enable_nan_detection=False, enable_p2p_check=False, triton_attention_reduce_in_fp32=False, triton_attention_num_kv_splits=8, num_continuous_decode_steps=1, delete_ckpt_after_loading=False, enable_memory_saver=False, allow_auto_truncate=False, enable_custom_logit_processor=False, tool_call_parser=None) 2025-05-17 22:53:05,699 - __main__ - WARNING - Attempt 7: Please wait for sglang server to become ready... 2025-05-17 22:53:06,767 - __main__ - WARNING - Attempt 8: Please wait for sglang server to become ready... 2025-05-17 22:53:07,823 - __main__ - WARNING - Attempt 9: Please wait for sglang server to become ready... 2025-05-17 22:53:08,864 - __main__ - WARNING - Attempt 10: Please wait for sglang server to become ready... 2025-05-17 22:53:09,904 - __main__ - WARNING - Attempt 11: Please wait for sglang server to become ready... 2025-05-17 22:53:10,937 - __main__ - WARNING - Attempt 12: Please wait for sglang server to become ready... 2025-05-17 22:53:11,990 - __main__ - WARNING - Attempt 13: Please wait for sglang server to become ready... 2025-05-17 22:53:13,053 - __main__ - WARNING - Attempt 14: Please wait for sglang server to become ready... 2025-05-17 22:53:14,099 - __main__ - WARNING - Attempt 15: Please wait for sglang server to become ready... 2025-05-17 22:53:14,330 - sglang - INFO - [2025-05-17 22:53:14] Use chat template for the OpenAI-compatible API server: qwen2-vl 2025-05-17 22:53:14,330 - __main__ - INFO - [2025-05-17 22:53:14] Use chat template for the OpenAI-compatible API server: qwen2-vl 2025-05-17 22:53:15,167 - __main__ - WARNING - Attempt 16: Please wait for sglang server to become ready... 2025-05-17 22:53:16,235 - __main__ - WARNING - Attempt 17: Please wait for sglang server to become ready... 2025-05-17 22:53:17,295 - __main__ - WARNING - Attempt 18: Please wait for sglang server to become ready... 2025-05-17 22:53:18,366 - __main__ - WARNING - Attempt 19: Please wait for sglang server to become ready... 2025-05-17 22:53:19,429 - __main__ - WARNING - Attempt 20: Please wait for sglang server to become ready... 2025-05-17 22:53:20,484 - __main__ - WARNING - Attempt 21: Please wait for sglang server to become ready... 2025-05-17 22:53:20,626 - sglang - INFO - [2025-05-17 22:53:20 TP0] Overlap scheduler is disabled for multimodal models. 2025-05-17 22:53:20,626 - __main__ - INFO - [2025-05-17 22:53:20 TP0] Overlap scheduler is disabled for multimodal models. 2025-05-17 22:53:21,154 - sglang - INFO - [2025-05-17 22:53:21 TP0] Automatically reduce --mem-fraction-static to 0.760 because this is a multimodal model. 2025-05-17 22:53:21,154 - __main__ - INFO - [2025-05-17 22:53:21 TP0] Automatically reduce --mem-fraction-static to 0.760 because this is a multimodal model. 2025-05-17 22:53:21,155 - sglang - INFO - [2025-05-17 22:53:21 TP0] Automatically turn off --chunked-prefill-size and disable radix cache for qwen2-vl. 2025-05-17 22:53:21,155 - __main__ - INFO - [2025-05-17 22:53:21 TP0] Automatically turn off --chunked-prefill-size and disable radix cache for qwen2-vl. 2025-05-17 22:53:21,155 - sglang - INFO - [2025-05-17 22:53:21 TP0] Init torch distributed begin. 2025-05-17 22:53:21,155 - __main__ - INFO - [2025-05-17 22:53:21 TP0] Init torch distributed begin. 2025-05-17 22:53:21,562 - __main__ - WARNING - Attempt 22: Please wait for sglang server to become ready... 2025-05-17 22:53:22,629 - __main__ - WARNING - Attempt 23: Please wait for sglang server to become ready... 2025-05-17 22:53:23,706 - __main__ - WARNING - Attempt 24: Please wait for sglang server to become ready... 2025-05-17 22:53:24,766 - __main__ - WARNING - Attempt 25: Please wait for sglang server to become ready... 2025-05-17 22:53:25,840 - __main__ - WARNING - Attempt 26: Please wait for sglang server to become ready... 2025-05-17 22:53:26,579 - sglang - INFO - [2025-05-17 22:53:26 TP0] Load weight begin. avail mem=23.33 GB 2025-05-17 22:53:26,580 - __main__ - INFO - [2025-05-17 22:53:26 TP0] Load weight begin. avail mem=23.33 GB 2025-05-17 22:53:26,916 - __main__ - WARNING - Attempt 27: Please wait for sglang server to become ready... 2025-05-17 22:53:27,637 - sglang - INFO - [2025-05-17 22:53:27 TP0] Using model weights format ['*.safetensors'] 2025-05-17 22:53:27,637 - __main__ - INFO - [2025-05-17 22:53:27 TP0] Using model weights format ['*.safetensors'] 2025-05-17 22:53:27,992 - __main__ - WARNING - Attempt 28: Please wait for sglang server to become ready... 2025-05-17 22:53:28,214 - sglang - INFO - Loading safetensors checkpoint shards: 0% Completed | 0/4 [00:00 - invalid_page rotation for olmocr_workspace/job_1747493917/input.pdf-15 2025-05-17 23:00:33,430 - __main__ - INFO - Built page query for olmocr_workspace/job_1747493917/input.pdf-15 2025-05-17 23:00:33,547 - sglang - INFO - [2025-05-17 23:00:33 TP0] Decode batch. #running-req: 3, #token: 9596, token usage: 0.25, gen throughput (token/s): 199.32, #queue-req: 0 2025-05-17 23:00:33,547 - __main__ - INFO - sglang running req: 3 queue req: 0 2025-05-17 23:00:33,648 - sglang - INFO - [2025-05-17 23:00:33 TP0] Prefill batch. #new-seq: 1, #new-token: 1868, #cached-token: 0, cache hit rate: 0.00%, token usage: 0.25, #running-req: 3, #queue-req: 0 2025-05-17 23:00:33,648 - __main__ - INFO - sglang running req: 3 queue req: 0 2025-05-17 23:00:35,157 - sglang - INFO - [2025-05-17 23:00:35 TP0] Decode batch. #running-req: 4, #token: 11620, token usage: 0.31, gen throughput (token/s): 96.88, #queue-req: 0 2025-05-17 23:00:35,158 - __main__ - INFO - sglang running req: 4 queue req: 0 2025-05-17 23:00:36,027 - sglang - INFO - [2025-05-17 23:00:36 TP0] Decode batch. #running-req: 4, #token: 11780, token usage: 0.31, gen throughput (token/s): 183.98, #queue-req: 0 2025-05-17 23:00:36,027 - __main__ - INFO - sglang running req: 4 queue req: 0 2025-05-17 23:00:36,903 - sglang - INFO - [2025-05-17 23:00:36 TP0] Decode batch. #running-req: 4, #token: 11940, token usage: 0.31, gen throughput (token/s): 182.74, #queue-req: 0 2025-05-17 23:00:36,903 - __main__ - INFO - sglang running req: 4 queue req: 0 2025-05-17 23:00:37,775 - sglang - INFO - [2025-05-17 23:00:37 TP0] Decode batch. #running-req: 4, #token: 12100, token usage: 0.32, gen throughput (token/s): 183.27, #queue-req: 0 2025-05-17 23:00:37,776 - __main__ - INFO - sglang running req: 4 queue req: 0 2025-05-17 23:00:38,285 - __main__ - INFO - Queue remaining: 0 2025-05-17 23:00:38,285 - __main__ - INFO - Metric Name Lifetime (tokens/sec) Recently (tokens/sec) ---------------------------------------------------------------------------------- sglang_input_tokens 230.12 230.12 sglang_output_tokens 60.90 60.90 2025-05-17 23:00:38,286 - __main__ - INFO - Worker ID | finished | started ----------+----------+-------- 0 | 12 | 15 2025-05-17 23:00:38,641 - sglang - INFO - [2025-05-17 23:00:38 TP0] Decode batch. #running-req: 3, #token: 8863, token usage: 0.23, gen throughput (token/s): 141.03, #queue-req: 0 2025-05-17 23:00:38,641 - __main__ - INFO - sglang running req: 3 queue req: 0 2025-05-17 23:00:39,505 - sglang - INFO - [2025-05-17 23:00:39 TP0] Decode batch. #running-req: 3, #token: 8983, token usage: 0.24, gen throughput (token/s): 138.86, #queue-req: 0 2025-05-17 23:00:39,505 - __main__ - INFO - sglang running req: 3 queue req: 0 2025-05-17 23:00:40,367 - sglang - INFO - [2025-05-17 23:00:40 TP0] Decode batch. #running-req: 2, #token: 5623, token usage: 0.15, gen throughput (token/s): 138.00, #queue-req: 0 2025-05-17 23:00:40,367 - __main__ - INFO - sglang running req: 2 queue req: 0 2025-05-17 23:00:41,219 - sglang - INFO - [2025-05-17 23:00:41 TP0] Decode batch. #running-req: 1, #token: 2184, token usage: 0.06, gen throughput (token/s): 79.84, #queue-req: 0 2025-05-17 23:00:41,219 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-05-17 23:00:42,057 - sglang - INFO - [2025-05-17 23:00:42 TP0] Decode batch. #running-req: 1, #token: 2224, token usage: 0.06, gen throughput (token/s): 47.74, #queue-req: 0 2025-05-17 23:00:42,057 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-05-17 23:00:42,892 - sglang - INFO - [2025-05-17 23:00:42 TP0] Decode batch. #running-req: 1, #token: 2264, token usage: 0.06, gen throughput (token/s): 47.90, #queue-req: 0 2025-05-17 23:00:42,892 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-05-17 23:00:43,727 - sglang - INFO - [2025-05-17 23:00:43 TP0] Decode batch. #running-req: 1, #token: 2304, token usage: 0.06, gen throughput (token/s): 47.87, #queue-req: 0 2025-05-17 23:00:43,728 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-05-17 23:00:44,566 - sglang - INFO - [2025-05-17 23:00:44 TP0] Decode batch. #running-req: 1, #token: 2344, token usage: 0.06, gen throughput (token/s): 47.68, #queue-req: 0 2025-05-17 23:00:44,566 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-05-17 23:00:45,409 - sglang - INFO - [2025-05-17 23:00:45 TP0] Decode batch. #running-req: 1, #token: 2384, token usage: 0.06, gen throughput (token/s): 47.48, #queue-req: 0 2025-05-17 23:00:45,409 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-05-17 23:00:46,004 - __main__ - INFO - Finished TaskGroup for worker on 02907a3ba6226f0399bbf3080296d8a1a280e502 2025-05-17 23:00:46,004 - __main__ - INFO - Got 1 docs for 02907a3ba6226f0399bbf3080296d8a1a280e502 2025-05-17 23:00:46,006 - __main__ - INFO - Worker 0 exiting due to empty queue 2025-05-17 23:00:46,006 - __main__ - INFO - Work done 2025-05-17 23:00:46,007 - __main__ - INFO - Got cancellation request for SGLang server 2025-05-17 23:06:23,302 - __main__ - INFO - Got --pdfs argument, going to add to the work queue 2025-05-17 23:06:23,302 - __main__ - INFO - Loading file at tests/gnarly_pdfs/badlines.pdf as PDF document 2025-05-17 23:06:23,302 - __main__ - INFO - Found 1 total pdf paths to add 2025-05-17 23:06:23,309 - __main__ - INFO - Calculated items_per_group: 50 based on average pages per PDF: 10.00 2025-05-17 23:06:23,548 - __main__ - INFO - Starting pipeline with PID 416546 2025-05-17 23:06:23,549 - __main__ - INFO - Downloading model with hugging face 'allenai/olmOCR-7B-0225-preview' 2025-05-17 23:06:29,154 - __main__ - WARNING - Attempt 1: Please wait for sglang server to become ready... 2025-05-17 23:06:30,200 - __main__ - WARNING - Attempt 2: Please wait for sglang server to become ready... 2025-05-17 23:06:31,251 - __main__ - WARNING - Attempt 3: Please wait for sglang server to become ready... 2025-05-17 23:06:32,316 - __main__ - WARNING - Attempt 4: Please wait for sglang server to become ready... 2025-05-17 23:06:33,381 - __main__ - WARNING - Attempt 5: Please wait for sglang server to become ready... 2025-05-17 23:06:34,451 - __main__ - WARNING - Attempt 6: Please wait for sglang server to become ready... 2025-05-17 23:06:35,178 - sglang - INFO - [2025-05-17 23:06:35] server_args=ServerArgs(model_path='allenai/olmOCR-7B-0225-preview', tokenizer_path='allenai/olmOCR-7B-0225-preview', tokenizer_mode='auto', load_format='auto', trust_remote_code=False, dtype='auto', kv_cache_dtype='auto', quantization_param_path=None, quantization=None, context_length=None, device='cuda', served_model_name='allenai/olmOCR-7B-0225-preview', chat_template='qwen2-vl', is_embedding=False, revision=None, skip_tokenizer_init=False, host='127.0.0.1', port=30024, mem_fraction_static=0.8, max_running_requests=None, max_total_tokens=None, chunked_prefill_size=2048, max_prefill_tokens=16384, schedule_policy='lpm', schedule_conservativeness=1.0, cpu_offload_gb=0, prefill_only_one_req=False, tp_size=1, stream_interval=1, stream_output=False, random_seed=153903282, constrained_json_whitespace_pattern=None, watchdog_timeout=300, download_dir=None, base_gpu_id=0, log_level='info', log_level_http='warning', log_requests=False, show_time_cost=False, enable_metrics=False, decode_log_interval=40, api_key=None, file_storage_pth='sglang_storage', enable_cache_report=False, dp_size=1, load_balance_method='round_robin', ep_size=1, dist_init_addr=None, nnodes=1, node_rank=0, json_model_override_args='{}', lora_paths=None, max_loras_per_batch=8, attention_backend='flashinfer', sampling_backend='flashinfer', grammar_backend='outlines', speculative_draft_model_path=None, speculative_algorithm=None, speculative_num_steps=5, speculative_num_draft_tokens=64, speculative_eagle_topk=8, enable_double_sparsity=False, ds_channel_config_path=None, ds_heavy_channel_num=32, ds_heavy_token_num=256, ds_heavy_channel_type='qk', ds_sparse_decode_threshold=4096, disable_radix_cache=False, disable_jump_forward=False, disable_cuda_graph=False, disable_cuda_graph_padding=False, disable_outlines_disk_cache=False, disable_custom_all_reduce=False, disable_mla=False, disable_overlap_schedule=False, enable_mixed_chunk=False, enable_dp_attention=False, enable_ep_moe=False, enable_torch_compile=False, torch_compile_max_bs=32, cuda_graph_max_bs=8, cuda_graph_bs=None, torchao_config='', enable_nan_detection=False, enable_p2p_check=False, triton_attention_reduce_in_fp32=False, triton_attention_num_kv_splits=8, num_continuous_decode_steps=1, delete_ckpt_after_loading=False, enable_memory_saver=False, allow_auto_truncate=False, enable_custom_logit_processor=False, tool_call_parser=None) 2025-05-17 23:06:35,178 - __main__ - INFO - [2025-05-17 23:06:35] server_args=ServerArgs(model_path='allenai/olmOCR-7B-0225-preview', tokenizer_path='allenai/olmOCR-7B-0225-preview', tokenizer_mode='auto', load_format='auto', trust_remote_code=False, dtype='auto', kv_cache_dtype='auto', quantization_param_path=None, quantization=None, context_length=None, device='cuda', served_model_name='allenai/olmOCR-7B-0225-preview', chat_template='qwen2-vl', is_embedding=False, revision=None, skip_tokenizer_init=False, host='127.0.0.1', port=30024, mem_fraction_static=0.8, max_running_requests=None, max_total_tokens=None, chunked_prefill_size=2048, max_prefill_tokens=16384, schedule_policy='lpm', schedule_conservativeness=1.0, cpu_offload_gb=0, prefill_only_one_req=False, tp_size=1, stream_interval=1, stream_output=False, random_seed=153903282, constrained_json_whitespace_pattern=None, watchdog_timeout=300, download_dir=None, base_gpu_id=0, log_level='info', log_level_http='warning', log_requests=False, show_time_cost=False, enable_metrics=False, decode_log_interval=40, api_key=None, file_storage_pth='sglang_storage', enable_cache_report=False, dp_size=1, load_balance_method='round_robin', ep_size=1, dist_init_addr=None, nnodes=1, node_rank=0, json_model_override_args='{}', lora_paths=None, max_loras_per_batch=8, attention_backend='flashinfer', sampling_backend='flashinfer', grammar_backend='outlines', speculative_draft_model_path=None, speculative_algorithm=None, speculative_num_steps=5, speculative_num_draft_tokens=64, speculative_eagle_topk=8, enable_double_sparsity=False, ds_channel_config_path=None, ds_heavy_channel_num=32, ds_heavy_token_num=256, ds_heavy_channel_type='qk', ds_sparse_decode_threshold=4096, disable_radix_cache=False, disable_jump_forward=False, disable_cuda_graph=False, disable_cuda_graph_padding=False, disable_outlines_disk_cache=False, disable_custom_all_reduce=False, disable_mla=False, disable_overlap_schedule=False, enable_mixed_chunk=False, enable_dp_attention=False, enable_ep_moe=False, enable_torch_compile=False, torch_compile_max_bs=32, cuda_graph_max_bs=8, cuda_graph_bs=None, torchao_config='', enable_nan_detection=False, enable_p2p_check=False, triton_attention_reduce_in_fp32=False, triton_attention_num_kv_splits=8, num_continuous_decode_steps=1, delete_ckpt_after_loading=False, enable_memory_saver=False, allow_auto_truncate=False, enable_custom_logit_processor=False, tool_call_parser=None) 2025-05-17 23:06:35,496 - __main__ - WARNING - Attempt 7: Please wait for sglang server to become ready... 2025-05-17 23:06:36,562 - __main__ - WARNING - Attempt 8: Please wait for sglang server to become ready... 2025-05-17 23:06:37,628 - __main__ - WARNING - Attempt 9: Please wait for sglang server to become ready... 2025-05-17 23:06:38,698 - __main__ - WARNING - Attempt 10: Please wait for sglang server to become ready... 2025-05-17 23:06:39,768 - __main__ - WARNING - Attempt 11: Please wait for sglang server to become ready... 2025-05-17 23:06:40,834 - __main__ - WARNING - Attempt 12: Please wait for sglang server to become ready... 2025-05-17 23:06:41,900 - __main__ - WARNING - Attempt 13: Please wait for sglang server to become ready... 2025-05-17 23:06:42,966 - __main__ - WARNING - Attempt 14: Please wait for sglang server to become ready... 2025-05-17 23:06:43,533 - sglang - INFO - [2025-05-17 23:06:43] Use chat template for the OpenAI-compatible API server: qwen2-vl 2025-05-17 23:06:43,533 - __main__ - INFO - [2025-05-17 23:06:43] Use chat template for the OpenAI-compatible API server: qwen2-vl 2025-05-17 23:06:44,045 - __main__ - WARNING - Attempt 15: Please wait for sglang server to become ready... 2025-05-17 23:06:45,117 - __main__ - WARNING - Attempt 16: Please wait for sglang server to become ready... 2025-05-17 23:06:46,182 - __main__ - WARNING - Attempt 17: Please wait for sglang server to become ready... 2025-05-17 23:06:47,236 - __main__ - WARNING - Attempt 18: Please wait for sglang server to become ready... 2025-05-17 23:06:48,297 - __main__ - WARNING - Attempt 19: Please wait for sglang server to become ready... 2025-05-17 23:06:49,361 - __main__ - WARNING - Attempt 20: Please wait for sglang server to become ready... 2025-05-17 23:06:49,771 - sglang - INFO - [2025-05-17 23:06:49 TP0] Overlap scheduler is disabled for multimodal models. 2025-05-17 23:06:49,771 - __main__ - INFO - [2025-05-17 23:06:49 TP0] Overlap scheduler is disabled for multimodal models. 2025-05-17 23:06:50,440 - __main__ - WARNING - Attempt 21: Please wait for sglang server to become ready... 2025-05-17 23:06:50,582 - sglang - INFO - [2025-05-17 23:06:50 TP0] Automatically reduce --mem-fraction-static to 0.760 because this is a multimodal model. 2025-05-17 23:06:50,582 - __main__ - INFO - [2025-05-17 23:06:50 TP0] Automatically reduce --mem-fraction-static to 0.760 because this is a multimodal model. 2025-05-17 23:06:50,583 - sglang - INFO - [2025-05-17 23:06:50 TP0] Automatically turn off --chunked-prefill-size and disable radix cache for qwen2-vl. 2025-05-17 23:06:50,583 - __main__ - INFO - [2025-05-17 23:06:50 TP0] Automatically turn off --chunked-prefill-size and disable radix cache for qwen2-vl. 2025-05-17 23:06:50,583 - sglang - INFO - [2025-05-17 23:06:50 TP0] Init torch distributed begin. 2025-05-17 23:06:50,583 - __main__ - INFO - [2025-05-17 23:06:50 TP0] Init torch distributed begin. 2025-05-17 23:06:51,520 - __main__ - WARNING - Attempt 22: Please wait for sglang server to become ready... 2025-05-17 23:06:52,595 - __main__ - WARNING - Attempt 23: Please wait for sglang server to become ready... 2025-05-17 23:06:53,661 - __main__ - WARNING - Attempt 24: Please wait for sglang server to become ready... 2025-05-17 23:06:54,727 - __main__ - WARNING - Attempt 25: Please wait for sglang server to become ready... 2025-05-17 23:06:55,780 - __main__ - WARNING - Attempt 26: Please wait for sglang server to become ready... 2025-05-17 23:06:55,881 - sglang - INFO - [2025-05-17 23:06:55 TP0] Load weight begin. avail mem=23.33 GB 2025-05-17 23:06:55,881 - __main__ - INFO - [2025-05-17 23:06:55 TP0] Load weight begin. avail mem=23.33 GB 2025-05-17 23:06:56,854 - __main__ - WARNING - Attempt 27: Please wait for sglang server to become ready... 2025-05-17 23:06:56,906 - sglang - INFO - [2025-05-17 23:06:56 TP0] Using model weights format ['*.safetensors'] 2025-05-17 23:06:56,906 - __main__ - INFO - [2025-05-17 23:06:56 TP0] Using model weights format ['*.safetensors'] 2025-05-17 23:06:57,540 - sglang - INFO - Loading safetensors checkpoint shards: 0% Completed | 0/4 [00:00 32768). Running this sequence through the model will result in indexing errors 2025-07-19 23:06:34,451 - __main__ - INFO - Process page tests/gnarly_pdfs/discoverworld_crazy_tables.pdf-26 cancelled 2025-07-19 23:06:34,451 - __main__ - INFO - Process page tests/gnarly_pdfs/discoverworld_crazy_tables.pdf-29 cancelled 2025-07-19 23:06:34,451 - __main__ - INFO - Process page tests/gnarly_pdfs/discoverworld_crazy_tables.pdf-8 cancelled 2025-07-19 23:06:34,451 - __main__ - INFO - Process page tests/gnarly_pdfs/discoverworld_crazy_tables.pdf-18 cancelled 2025-07-19 23:06:34,452 - __main__ - INFO - Process page tests/gnarly_pdfs/discoverworld_crazy_tables.pdf-9 cancelled 2025-07-19 23:06:34,452 - __main__ - INFO - Process page tests/gnarly_pdfs/discoverworld_crazy_tables.pdf-19 cancelled 2025-07-19 23:06:34,452 - __main__ - INFO - Process page tests/gnarly_pdfs/discoverworld_crazy_tables.pdf-10 cancelled 2025-07-19 23:06:34,452 - __main__ - INFO - Process page tests/gnarly_pdfs/discoverworld_crazy_tables.pdf-20 cancelled 2025-07-19 23:06:34,452 - __main__ - INFO - Process page tests/gnarly_pdfs/discoverworld_crazy_tables.pdf-11 cancelled 2025-07-19 23:06:34,452 - __main__ - INFO - Process page tests/gnarly_pdfs/discoverworld_crazy_tables.pdf-21 cancelled 2025-07-19 23:06:34,452 - __main__ - INFO - Process page tests/gnarly_pdfs/discoverworld_crazy_tables.pdf-1 cancelled 2025-07-19 23:06:34,452 - __main__ - INFO - Process page tests/gnarly_pdfs/discoverworld_crazy_tables.pdf-12 cancelled 2025-07-19 23:06:34,452 - __main__ - INFO - Process page tests/gnarly_pdfs/discoverworld_crazy_tables.pdf-22 cancelled 2025-07-19 23:06:34,452 - __main__ - INFO - Process page tests/gnarly_pdfs/discoverworld_crazy_tables.pdf-2 cancelled 2025-07-19 23:06:34,452 - __main__ - INFO - Process page tests/gnarly_pdfs/discoverworld_crazy_tables.pdf-13 cancelled 2025-07-19 23:06:34,452 - __main__ - INFO - Process page tests/gnarly_pdfs/discoverworld_crazy_tables.pdf-23 cancelled 2025-07-19 23:06:34,452 - __main__ - INFO - Process page tests/gnarly_pdfs/discoverworld_crazy_tables.pdf-3 cancelled 2025-07-19 23:06:34,452 - __main__ - INFO - Process page tests/gnarly_pdfs/discoverworld_crazy_tables.pdf-14 cancelled 2025-07-19 23:06:34,533 - __main__ - INFO - Process page tests/gnarly_pdfs/discoverworld_crazy_tables.pdf-24 cancelled 2025-07-19 23:06:34,533 - __main__ - INFO - Process page tests/gnarly_pdfs/discoverworld_crazy_tables.pdf-4 cancelled 2025-07-19 23:06:34,533 - __main__ - INFO - Process page tests/gnarly_pdfs/discoverworld_crazy_tables.pdf-15 cancelled 2025-07-19 23:06:34,533 - __main__ - INFO - Process page tests/gnarly_pdfs/discoverworld_crazy_tables.pdf-25 cancelled 2025-07-19 23:06:34,533 - __main__ - INFO - Process page tests/gnarly_pdfs/discoverworld_crazy_tables.pdf-5 cancelled 2025-07-19 23:06:34,533 - __main__ - INFO - Process page tests/gnarly_pdfs/discoverworld_crazy_tables.pdf-16 cancelled 2025-07-19 23:06:34,533 - __main__ - INFO - Process page tests/gnarly_pdfs/discoverworld_crazy_tables.pdf-27 cancelled 2025-07-19 23:06:34,533 - __main__ - INFO - Process page tests/gnarly_pdfs/discoverworld_crazy_tables.pdf-6 cancelled 2025-07-19 23:06:34,533 - __main__ - INFO - Process page tests/gnarly_pdfs/discoverworld_crazy_tables.pdf-17 cancelled 2025-07-19 23:06:34,533 - __main__ - INFO - Process page tests/gnarly_pdfs/discoverworld_crazy_tables.pdf-28 cancelled 2025-07-19 23:06:34,534 - __main__ - INFO - Process page tests/gnarly_pdfs/discoverworld_crazy_tables.pdf-7 cancelled 2025-07-19 23:06:34,534 - __main__ - INFO - Process page tests/gnarly_pdfs/ti89_guidebook_programming.pdf-40 cancelled 2025-07-19 23:06:34,534 - __main__ - INFO - Process page tests/gnarly_pdfs/ti89_guidebook_programming.pdf-18 cancelled 2025-07-19 23:06:34,534 - __main__ - INFO - Process page tests/gnarly_pdfs/ti89_guidebook_programming.pdf-9 cancelled 2025-07-19 23:06:34,534 - __main__ - INFO - Process page tests/gnarly_pdfs/ti89_guidebook_programming.pdf-30 cancelled 2025-07-19 23:06:34,534 - __main__ - INFO - Process page tests/gnarly_pdfs/ti89_guidebook_programming.pdf-19 cancelled 2025-07-19 23:06:34,534 - __main__ - INFO - Process page tests/gnarly_pdfs/ti89_guidebook_programming.pdf-10 cancelled 2025-07-19 23:06:34,534 - __main__ - INFO - Process page tests/gnarly_pdfs/ti89_guidebook_programming.pdf-31 cancelled 2025-07-19 23:06:34,534 - __main__ - INFO - Process page tests/gnarly_pdfs/ti89_guidebook_programming.pdf-20 cancelled 2025-07-19 23:06:34,534 - __main__ - INFO - Process page tests/gnarly_pdfs/ti89_guidebook_programming.pdf-11 cancelled 2025-07-19 23:06:34,534 - __main__ - INFO - Process page tests/gnarly_pdfs/ti89_guidebook_programming.pdf-32 cancelled 2025-07-19 23:06:34,534 - __main__ - INFO - Process page tests/gnarly_pdfs/ti89_guidebook_programming.pdf-1 cancelled 2025-07-19 23:06:34,534 - __main__ - INFO - Process page tests/gnarly_pdfs/ti89_guidebook_programming.pdf-21 cancelled 2025-07-19 23:06:34,534 - __main__ - INFO - Process page tests/gnarly_pdfs/ti89_guidebook_programming.pdf-12 cancelled 2025-07-19 23:06:34,534 - __main__ - INFO - Process page tests/gnarly_pdfs/ti89_guidebook_programming.pdf-33 cancelled 2025-07-19 23:06:34,535 - __main__ - INFO - Process page tests/gnarly_pdfs/ti89_guidebook_programming.pdf-2 cancelled 2025-07-19 23:06:34,535 - __main__ - INFO - Process page tests/gnarly_pdfs/ti89_guidebook_programming.pdf-22 cancelled 2025-07-19 23:06:34,535 - __main__ - INFO - Process page tests/gnarly_pdfs/ti89_guidebook_programming.pdf-13 cancelled 2025-07-19 23:06:34,535 - __main__ - INFO - Process page tests/gnarly_pdfs/ti89_guidebook_programming.pdf-34 cancelled 2025-07-19 23:06:34,535 - __main__ - INFO - Process page tests/gnarly_pdfs/ti89_guidebook_programming.pdf-3 cancelled 2025-07-19 23:06:34,535 - __main__ - INFO - Process page tests/gnarly_pdfs/ti89_guidebook_programming.pdf-23 cancelled 2025-07-19 23:06:34,535 - __main__ - INFO - Process page tests/gnarly_pdfs/ti89_guidebook_programming.pdf-14 cancelled 2025-07-19 23:06:34,535 - __main__ - INFO - Process page tests/gnarly_pdfs/ti89_guidebook_programming.pdf-35 cancelled 2025-07-19 23:06:34,535 - __main__ - INFO - Process page tests/gnarly_pdfs/ti89_guidebook_programming.pdf-4 cancelled 2025-07-19 23:06:34,535 - __main__ - INFO - Process page tests/gnarly_pdfs/ti89_guidebook_programming.pdf-24 cancelled 2025-07-19 23:06:34,535 - __main__ - INFO - Process page tests/gnarly_pdfs/ti89_guidebook_programming.pdf-15 cancelled 2025-07-19 23:06:34,535 - __main__ - INFO - Process page tests/gnarly_pdfs/ti89_guidebook_programming.pdf-36 cancelled 2025-07-19 23:06:34,535 - __main__ - INFO - Process page tests/gnarly_pdfs/ti89_guidebook_programming.pdf-5 cancelled 2025-07-19 23:06:34,535 - __main__ - INFO - Process page tests/gnarly_pdfs/ti89_guidebook_programming.pdf-25 cancelled 2025-07-19 23:06:34,535 - __main__ - INFO - Process page tests/gnarly_pdfs/ti89_guidebook_programming.pdf-16 cancelled 2025-07-19 23:06:34,535 - __main__ - INFO - Process page tests/gnarly_pdfs/ti89_guidebook_programming.pdf-37 cancelled 2025-07-19 23:06:34,535 - __main__ - INFO - Process page tests/gnarly_pdfs/ti89_guidebook_programming.pdf-6 cancelled 2025-07-19 23:06:34,536 - __main__ - INFO - Process page tests/gnarly_pdfs/ti89_guidebook_programming.pdf-27 cancelled 2025-07-19 23:06:34,536 - __main__ - INFO - Process page tests/gnarly_pdfs/ti89_guidebook_programming.pdf-17 cancelled 2025-07-19 23:06:34,536 - __main__ - INFO - Process page tests/gnarly_pdfs/ti89_guidebook_programming.pdf-38 cancelled 2025-07-19 23:06:34,536 - __main__ - INFO - Process page tests/gnarly_pdfs/ti89_guidebook_programming.pdf-7 cancelled 2025-07-19 23:06:34,536 - __main__ - INFO - Process page tests/gnarly_pdfs/ti89_guidebook_programming.pdf-28 cancelled 2025-07-19 23:06:34,536 - __main__ - INFO - Process page tests/gnarly_pdfs/ti89_guidebook_programming.pdf-29 cancelled 2025-07-19 23:06:34,536 - __main__ - INFO - Process page tests/gnarly_pdfs/ti89_guidebook_programming.pdf-26 cancelled 2025-07-19 23:06:34,536 - __main__ - INFO - Process page tests/gnarly_pdfs/ti89_guidebook_programming.pdf-39 cancelled 2025-07-19 23:06:34,536 - __main__ - INFO - Process page tests/gnarly_pdfs/ti89_guidebook_programming.pdf-8 cancelled 2025-07-19 23:06:34,536 - __main__ - INFO - Process page tests/gnarly_pdfs/not_parsing.pdf-1 cancelled 2025-07-19 23:06:34,536 - __main__ - INFO - Process page tests/gnarly_pdfs/not_parsing.pdf-4 cancelled 2025-07-19 23:06:34,536 - __main__ - INFO - Process page tests/gnarly_pdfs/not_parsing.pdf-7 cancelled 2025-07-19 23:06:34,536 - __main__ - INFO - Process page tests/gnarly_pdfs/not_parsing.pdf-2 cancelled 2025-07-19 23:06:34,536 - __main__ - INFO - Process page tests/gnarly_pdfs/not_parsing.pdf-5 cancelled 2025-07-19 23:06:34,537 - __main__ - INFO - Process page tests/gnarly_pdfs/not_parsing.pdf-8 cancelled 2025-07-19 23:06:34,537 - __main__ - INFO - Process page tests/gnarly_pdfs/not_parsing.pdf-3 cancelled 2025-07-19 23:06:34,537 - __main__ - INFO - Process page tests/gnarly_pdfs/not_parsing.pdf-6 cancelled 2025-07-19 23:06:34,537 - __main__ - INFO - Process page tests/gnarly_pdfs/large_prompt_hint2.pdf-5 cancelled 2025-07-19 23:06:34,537 - __main__ - INFO - Process page tests/gnarly_pdfs/large_prompt_hint2.pdf-3 cancelled 2025-07-19 23:06:34,537 - __main__ - INFO - Process page tests/gnarly_pdfs/large_prompt_hint2.pdf-6 cancelled 2025-07-19 23:06:34,537 - __main__ - INFO - Process page tests/gnarly_pdfs/large_prompt_hint2.pdf-1 cancelled 2025-07-19 23:06:34,537 - __main__ - INFO - Process page tests/gnarly_pdfs/large_prompt_hint2.pdf-4 cancelled 2025-07-19 23:06:34,537 - __main__ - INFO - Process page tests/gnarly_pdfs/large_prompt_hint2.pdf-2 cancelled 2025-07-19 23:06:34,537 - __main__ - INFO - Process page tests/gnarly_pdfs/failing_anchor_pg4.pdf-1 cancelled 2025-07-19 23:06:34,537 - __main__ - INFO - Process page tests/gnarly_pdfs/failing_anchor_pg4.pdf-4 cancelled 2025-07-19 23:06:34,537 - __main__ - INFO - Process page tests/gnarly_pdfs/failing_anchor_pg4.pdf-7 cancelled 2025-07-19 23:06:34,537 - __main__ - INFO - Process page tests/gnarly_pdfs/failing_anchor_pg4.pdf-2 cancelled 2025-07-19 23:06:34,537 - __main__ - INFO - Process page tests/gnarly_pdfs/failing_anchor_pg4.pdf-5 cancelled 2025-07-19 23:06:34,537 - __main__ - INFO - Process page tests/gnarly_pdfs/failing_anchor_pg4.pdf-8 cancelled 2025-07-19 23:06:34,538 - __main__ - INFO - Process page tests/gnarly_pdfs/failing_anchor_pg4.pdf-3 cancelled 2025-07-19 23:06:34,538 - __main__ - INFO - Process page tests/gnarly_pdfs/failing_anchor_pg4.pdf-6 cancelled 2025-07-19 23:06:34,538 - __main__ - INFO - Process page tests/gnarly_pdfs/failing_pdf_pg9.pdf-8 cancelled 2025-07-19 23:06:34,538 - __main__ - INFO - Process page tests/gnarly_pdfs/failing_pdf_pg9.pdf-3 cancelled 2025-07-19 23:06:34,538 - __main__ - INFO - Process page tests/gnarly_pdfs/failing_pdf_pg9.pdf-6 cancelled 2025-07-19 23:06:34,538 - __main__ - INFO - Process page tests/gnarly_pdfs/failing_pdf_pg9.pdf-1 cancelled 2025-07-19 23:06:34,538 - __main__ - INFO - Process page tests/gnarly_pdfs/failing_pdf_pg9.pdf-9 cancelled 2025-07-19 23:06:34,538 - __main__ - INFO - Process page tests/gnarly_pdfs/failing_pdf_pg9.pdf-4 cancelled 2025-07-19 23:06:34,538 - __main__ - INFO - Process page tests/gnarly_pdfs/failing_pdf_pg9.pdf-7 cancelled 2025-07-19 23:06:34,538 - __main__ - INFO - Process page tests/gnarly_pdfs/failing_pdf_pg9.pdf-2 cancelled 2025-07-19 23:06:34,538 - __main__ - INFO - Process page tests/gnarly_pdfs/failing_pdf_pg9.pdf-5 cancelled 2025-07-19 23:06:34,538 - __main__ - INFO - Process page tests/gnarly_pdfs/not_parsing2.pdf-5 cancelled 2025-07-19 23:06:34,538 - __main__ - INFO - Process page tests/gnarly_pdfs/not_parsing2.pdf-8 cancelled 2025-07-19 23:06:34,538 - __main__ - INFO - Process page tests/gnarly_pdfs/not_parsing2.pdf-3 cancelled 2025-07-19 23:06:34,538 - __main__ - INFO - Process page tests/gnarly_pdfs/not_parsing2.pdf-6 cancelled 2025-07-19 23:06:34,539 - __main__ - INFO - Process page tests/gnarly_pdfs/not_parsing2.pdf-1 cancelled 2025-07-19 23:06:34,539 - __main__ - INFO - Process page tests/gnarly_pdfs/not_parsing2.pdf-9 cancelled 2025-07-19 23:06:34,539 - __main__ - INFO - Process page tests/gnarly_pdfs/not_parsing2.pdf-4 cancelled 2025-07-19 23:06:34,539 - __main__ - INFO - Process page tests/gnarly_pdfs/not_parsing2.pdf-7 cancelled 2025-07-19 23:06:34,539 - __main__ - INFO - Process page tests/gnarly_pdfs/not_parsing2.pdf-2 cancelled 2025-07-19 23:06:34,539 - __main__ - INFO - Process page tests/gnarly_pdfs/ambiguous.pdf-1 cancelled 2025-07-19 23:06:34,539 - __main__ - INFO - Process page tests/gnarly_pdfs/bws_book_ch2.pdf-26 cancelled 2025-07-19 23:06:34,539 - __main__ - INFO - Process page tests/gnarly_pdfs/bws_book_ch2.pdf-29 cancelled 2025-07-19 23:06:34,539 - __main__ - INFO - Process page tests/gnarly_pdfs/bws_book_ch2.pdf-8 cancelled 2025-07-19 23:06:34,539 - __main__ - INFO - Process page tests/gnarly_pdfs/bws_book_ch2.pdf-40 cancelled 2025-07-19 23:06:34,539 - __main__ - INFO - Process page tests/gnarly_pdfs/bws_book_ch2.pdf-18 cancelled 2025-07-19 23:06:34,539 - __main__ - INFO - Process page tests/gnarly_pdfs/bws_book_ch2.pdf-30 cancelled 2025-07-19 23:06:34,539 - __main__ - INFO - Process page tests/gnarly_pdfs/bws_book_ch2.pdf-9 cancelled 2025-07-19 23:06:34,539 - __main__ - INFO - Process page tests/gnarly_pdfs/bws_book_ch2.pdf-41 cancelled 2025-07-19 23:06:34,539 - __main__ - INFO - Process page tests/gnarly_pdfs/bws_book_ch2.pdf-19 cancelled 2025-07-19 23:06:34,539 - __main__ - INFO - Process page tests/gnarly_pdfs/bws_book_ch2.pdf-31 cancelled 2025-07-19 23:06:34,540 - __main__ - INFO - Process page tests/gnarly_pdfs/bws_book_ch2.pdf-10 cancelled 2025-07-19 23:06:34,540 - __main__ - INFO - Process page tests/gnarly_pdfs/bws_book_ch2.pdf-42 cancelled 2025-07-19 23:06:34,540 - __main__ - INFO - Process page tests/gnarly_pdfs/bws_book_ch2.pdf-20 cancelled 2025-07-19 23:06:34,540 - __main__ - INFO - Process page tests/gnarly_pdfs/bws_book_ch2.pdf-32 cancelled 2025-07-19 23:06:34,540 - __main__ - INFO - Process page tests/gnarly_pdfs/bws_book_ch2.pdf-11 cancelled 2025-07-19 23:06:34,540 - __main__ - INFO - Process page tests/gnarly_pdfs/bws_book_ch2.pdf-43 cancelled 2025-07-19 23:06:34,540 - __main__ - INFO - Process page tests/gnarly_pdfs/bws_book_ch2.pdf-21 cancelled 2025-07-19 23:06:34,540 - __main__ - INFO - Process page tests/gnarly_pdfs/bws_book_ch2.pdf-1 cancelled 2025-07-19 23:06:34,540 - __main__ - INFO - Process page tests/gnarly_pdfs/bws_book_ch2.pdf-33 cancelled 2025-07-19 23:06:34,540 - __main__ - INFO - Process page tests/gnarly_pdfs/bws_book_ch2.pdf-12 cancelled 2025-07-19 23:06:34,540 - __main__ - INFO - Process page tests/gnarly_pdfs/bws_book_ch2.pdf-44 cancelled 2025-07-19 23:06:34,540 - __main__ - INFO - Process page tests/gnarly_pdfs/bws_book_ch2.pdf-22 cancelled 2025-07-19 23:06:34,540 - __main__ - INFO - Process page tests/gnarly_pdfs/bws_book_ch2.pdf-2 cancelled 2025-07-19 23:06:34,540 - __main__ - INFO - Process page tests/gnarly_pdfs/bws_book_ch2.pdf-34 cancelled 2025-07-19 23:06:34,540 - __main__ - INFO - Process page tests/gnarly_pdfs/bws_book_ch2.pdf-13 cancelled 2025-07-19 23:06:34,541 - __main__ - INFO - Process page tests/gnarly_pdfs/bws_book_ch2.pdf-45 cancelled 2025-07-19 23:06:34,541 - __main__ - INFO - Process page tests/gnarly_pdfs/bws_book_ch2.pdf-23 cancelled 2025-07-19 23:06:34,541 - __main__ - INFO - Process page tests/gnarly_pdfs/bws_book_ch2.pdf-3 cancelled 2025-07-19 23:06:34,541 - __main__ - INFO - Process page tests/gnarly_pdfs/bws_book_ch2.pdf-35 cancelled 2025-07-19 23:06:34,541 - __main__ - INFO - Process page tests/gnarly_pdfs/bws_book_ch2.pdf-14 cancelled 2025-07-19 23:06:34,541 - __main__ - INFO - Process page tests/gnarly_pdfs/bws_book_ch2.pdf-46 cancelled 2025-07-19 23:06:34,541 - __main__ - INFO - Process page tests/gnarly_pdfs/bws_book_ch2.pdf-24 cancelled 2025-07-19 23:06:34,541 - __main__ - INFO - Process page tests/gnarly_pdfs/bws_book_ch2.pdf-4 cancelled 2025-07-19 23:06:34,541 - __main__ - INFO - Process page tests/gnarly_pdfs/bws_book_ch2.pdf-36 cancelled 2025-07-19 23:06:34,541 - __main__ - INFO - Process page tests/gnarly_pdfs/bws_book_ch2.pdf-15 cancelled 2025-07-19 23:06:34,541 - __main__ - INFO - Process page tests/gnarly_pdfs/bws_book_ch2.pdf-39 cancelled 2025-07-19 23:06:34,541 - __main__ - INFO - Process page tests/gnarly_pdfs/bws_book_ch2.pdf-47 cancelled 2025-07-19 23:06:34,541 - __main__ - INFO - Process page tests/gnarly_pdfs/bws_book_ch2.pdf-25 cancelled 2025-07-19 23:06:34,541 - __main__ - INFO - Process page tests/gnarly_pdfs/bws_book_ch2.pdf-5 cancelled 2025-07-19 23:06:34,542 - __main__ - INFO - Process page tests/gnarly_pdfs/bws_book_ch2.pdf-37 cancelled 2025-07-19 23:06:34,542 - __main__ - INFO - Process page tests/gnarly_pdfs/bws_book_ch2.pdf-16 cancelled 2025-07-19 23:06:34,542 - __main__ - INFO - Process page tests/gnarly_pdfs/bws_book_ch2.pdf-48 cancelled 2025-07-19 23:06:34,542 - __main__ - INFO - Process page tests/gnarly_pdfs/bws_book_ch2.pdf-27 cancelled 2025-07-19 23:06:34,542 - __main__ - INFO - Process page tests/gnarly_pdfs/bws_book_ch2.pdf-6 cancelled 2025-07-19 23:06:34,542 - __main__ - INFO - Process page tests/gnarly_pdfs/bws_book_ch2.pdf-38 cancelled 2025-07-19 23:06:34,542 - __main__ - INFO - Process page tests/gnarly_pdfs/bws_book_ch2.pdf-17 cancelled 2025-07-19 23:06:34,542 - __main__ - INFO - Process page tests/gnarly_pdfs/bws_book_ch2.pdf-28 cancelled 2025-07-19 23:06:34,542 - __main__ - INFO - Process page tests/gnarly_pdfs/bws_book_ch2.pdf-7 cancelled 2025-07-19 23:06:34,542 - __main__ - INFO - Process page tests/gnarly_pdfs/edgar.pdf-1 cancelled 2025-07-19 23:06:34,542 - __main__ - INFO - Process page tests/gnarly_pdfs/olmo-page-1.pdf-1 cancelled 2025-07-19 23:06:34,542 - __main__ - INFO - Process page tests/gnarly_pdfs/dolma-page-1.pdf-1 cancelled 2025-07-19 23:06:34,542 - __main__ - INFO - Process page tests/gnarly_pdfs/overrun_on_pg8.pdf-31 cancelled 2025-07-19 23:06:34,543 - __main__ - INFO - Process page tests/gnarly_pdfs/overrun_on_pg8.pdf-10 cancelled 2025-07-19 23:06:34,543 - __main__ - INFO - Process page tests/gnarly_pdfs/overrun_on_pg8.pdf-42 cancelled 2025-07-19 23:06:34,543 - __main__ - INFO - Process page tests/gnarly_pdfs/overrun_on_pg8.pdf-20 cancelled 2025-07-19 23:06:34,543 - __main__ - INFO - Process page tests/gnarly_pdfs/overrun_on_pg8.pdf-53 cancelled 2025-07-19 23:06:34,543 - __main__ - INFO - Process page tests/gnarly_pdfs/overrun_on_pg8.pdf-32 cancelled 2025-07-19 23:06:34,543 - __main__ - INFO - Process page tests/gnarly_pdfs/overrun_on_pg8.pdf-11 cancelled 2025-07-19 23:06:34,543 - __main__ - INFO - Process page tests/gnarly_pdfs/overrun_on_pg8.pdf-43 cancelled 2025-07-19 23:06:34,543 - __main__ - INFO - Process page tests/gnarly_pdfs/overrun_on_pg8.pdf-1 cancelled 2025-07-19 23:06:34,543 - __main__ - INFO - Process page tests/gnarly_pdfs/overrun_on_pg8.pdf-21 cancelled 2025-07-19 23:06:34,543 - __main__ - INFO - Process page tests/gnarly_pdfs/overrun_on_pg8.pdf-54 cancelled 2025-07-19 23:06:34,543 - __main__ - INFO - Process page tests/gnarly_pdfs/overrun_on_pg8.pdf-33 cancelled 2025-07-19 23:06:34,543 - __main__ - INFO - Process page tests/gnarly_pdfs/overrun_on_pg8.pdf-12 cancelled 2025-07-19 23:06:34,543 - __main__ - INFO - Process page tests/gnarly_pdfs/overrun_on_pg8.pdf-44 cancelled 2025-07-19 23:06:34,543 - __main__ - INFO - Process page tests/gnarly_pdfs/overrun_on_pg8.pdf-2 cancelled 2025-07-19 23:06:34,543 - __main__ - INFO - Process page tests/gnarly_pdfs/overrun_on_pg8.pdf-22 cancelled 2025-07-19 23:06:34,544 - __main__ - INFO - Process page tests/gnarly_pdfs/overrun_on_pg8.pdf-34 cancelled 2025-07-19 23:06:34,544 - __main__ - INFO - Process page tests/gnarly_pdfs/overrun_on_pg8.pdf-13 cancelled 2025-07-19 23:06:34,544 - __main__ - INFO - Process page tests/gnarly_pdfs/overrun_on_pg8.pdf-45 cancelled 2025-07-19 23:06:34,544 - __main__ - INFO - Process page tests/gnarly_pdfs/overrun_on_pg8.pdf-3 cancelled 2025-07-19 23:06:34,544 - __main__ - INFO - Process page tests/gnarly_pdfs/overrun_on_pg8.pdf-23 cancelled 2025-07-19 23:06:34,544 - __main__ - INFO - Process page tests/gnarly_pdfs/overrun_on_pg8.pdf-35 cancelled 2025-07-19 23:06:34,544 - __main__ - INFO - Process page tests/gnarly_pdfs/overrun_on_pg8.pdf-14 cancelled 2025-07-19 23:06:34,544 - __main__ - INFO - Process page tests/gnarly_pdfs/overrun_on_pg8.pdf-46 cancelled 2025-07-19 23:06:34,544 - __main__ - INFO - Process page tests/gnarly_pdfs/overrun_on_pg8.pdf-4 cancelled 2025-07-19 23:06:34,544 - __main__ - INFO - Process page tests/gnarly_pdfs/overrun_on_pg8.pdf-24 cancelled 2025-07-19 23:06:34,544 - __main__ - INFO - Process page tests/gnarly_pdfs/overrun_on_pg8.pdf-36 cancelled 2025-07-19 23:06:34,544 - __main__ - INFO - Process page tests/gnarly_pdfs/overrun_on_pg8.pdf-52 cancelled 2025-07-19 23:06:34,544 - __main__ - INFO - Process page tests/gnarly_pdfs/overrun_on_pg8.pdf-15 cancelled 2025-07-19 23:06:34,544 - __main__ - INFO - Process page tests/gnarly_pdfs/overrun_on_pg8.pdf-47 cancelled 2025-07-19 23:06:34,545 - __main__ - INFO - Process page tests/gnarly_pdfs/overrun_on_pg8.pdf-5 cancelled 2025-07-19 23:06:34,545 - __main__ - INFO - Process page tests/gnarly_pdfs/overrun_on_pg8.pdf-25 cancelled 2025-07-19 23:06:34,545 - __main__ - INFO - Process page tests/gnarly_pdfs/overrun_on_pg8.pdf-37 cancelled 2025-07-19 23:06:34,545 - __main__ - INFO - Process page tests/gnarly_pdfs/overrun_on_pg8.pdf-16 cancelled 2025-07-19 23:06:34,545 - __main__ - INFO - Process page tests/gnarly_pdfs/overrun_on_pg8.pdf-48 cancelled 2025-07-19 23:06:34,545 - __main__ - INFO - Process page tests/gnarly_pdfs/overrun_on_pg8.pdf-6 cancelled 2025-07-19 23:06:34,545 - __main__ - INFO - Process page tests/gnarly_pdfs/overrun_on_pg8.pdf-27 cancelled 2025-07-19 23:06:34,545 - __main__ - INFO - Process page tests/gnarly_pdfs/overrun_on_pg8.pdf-38 cancelled 2025-07-19 23:06:34,545 - __main__ - INFO - Process page tests/gnarly_pdfs/overrun_on_pg8.pdf-17 cancelled 2025-07-19 23:06:34,545 - __main__ - INFO - Process page tests/gnarly_pdfs/overrun_on_pg8.pdf-49 cancelled 2025-07-19 23:06:34,545 - __main__ - INFO - Process page tests/gnarly_pdfs/overrun_on_pg8.pdf-7 cancelled 2025-07-19 23:06:34,545 - __main__ - INFO - Process page tests/gnarly_pdfs/overrun_on_pg8.pdf-28 cancelled 2025-07-19 23:06:34,545 - __main__ - INFO - Process page tests/gnarly_pdfs/overrun_on_pg8.pdf-39 cancelled 2025-07-19 23:06:34,545 - __main__ - INFO - Process page tests/gnarly_pdfs/overrun_on_pg8.pdf-26 cancelled 2025-07-19 23:06:34,545 - __main__ - INFO - Process page tests/gnarly_pdfs/overrun_on_pg8.pdf-50 cancelled 2025-07-19 23:06:34,545 - __main__ - INFO - Process page tests/gnarly_pdfs/overrun_on_pg8.pdf-8 cancelled 2025-07-19 23:06:34,545 - __main__ - INFO - Process page tests/gnarly_pdfs/overrun_on_pg8.pdf-29 cancelled 2025-07-19 23:06:34,546 - __main__ - INFO - Process page tests/gnarly_pdfs/overrun_on_pg8.pdf-40 cancelled 2025-07-19 23:06:34,546 - __main__ - INFO - Process page tests/gnarly_pdfs/overrun_on_pg8.pdf-18 cancelled 2025-07-19 23:06:34,546 - __main__ - INFO - Process page tests/gnarly_pdfs/overrun_on_pg8.pdf-51 cancelled 2025-07-19 23:06:34,546 - __main__ - INFO - Process page tests/gnarly_pdfs/overrun_on_pg8.pdf-9 cancelled 2025-07-19 23:06:34,546 - __main__ - INFO - Process page tests/gnarly_pdfs/overrun_on_pg8.pdf-30 cancelled 2025-07-19 23:06:34,546 - __main__ - INFO - Process page tests/gnarly_pdfs/overrun_on_pg8.pdf-41 cancelled 2025-07-19 23:06:34,546 - __main__ - INFO - Process page tests/gnarly_pdfs/overrun_on_pg8.pdf-19 cancelled 2025-07-19 23:06:34,546 - __main__ - INFO - Process page tests/gnarly_pdfs/lots_of_chem_tables.pdf-4 cancelled 2025-07-19 23:06:34,546 - __main__ - INFO - Process page tests/gnarly_pdfs/lots_of_chem_tables.pdf-7 cancelled 2025-07-19 23:06:34,546 - __main__ - INFO - Process page tests/gnarly_pdfs/lots_of_chem_tables.pdf-2 cancelled 2025-07-19 23:06:34,546 - __main__ - INFO - Process page tests/gnarly_pdfs/lots_of_chem_tables.pdf-5 cancelled 2025-07-19 23:06:34,546 - __main__ - INFO - Process page tests/gnarly_pdfs/lots_of_chem_tables.pdf-8 cancelled 2025-07-19 23:06:34,546 - __main__ - INFO - Process page tests/gnarly_pdfs/lots_of_chem_tables.pdf-3 cancelled 2025-07-19 23:06:34,546 - __main__ - INFO - Process page tests/gnarly_pdfs/lots_of_chem_tables.pdf-9 cancelled 2025-07-19 23:06:34,546 - __main__ - INFO - Process page tests/gnarly_pdfs/lots_of_chem_tables.pdf-6 cancelled 2025-07-19 23:06:34,546 - __main__ - INFO - Process page tests/gnarly_pdfs/lots_of_chem_tables.pdf-1 cancelled 2025-07-19 23:06:34,546 - __main__ - INFO - Process page tests/gnarly_pdfs/lots_of_sci_tables.pdf-6 cancelled 2025-07-19 23:06:34,547 - __main__ - INFO - Process page tests/gnarly_pdfs/lots_of_sci_tables.pdf-1 cancelled 2025-07-19 23:06:34,547 - __main__ - INFO - Process page tests/gnarly_pdfs/lots_of_sci_tables.pdf-4 cancelled 2025-07-19 23:06:34,547 - __main__ - INFO - Process page tests/gnarly_pdfs/lots_of_sci_tables.pdf-2 cancelled 2025-07-19 23:06:34,547 - __main__ - INFO - Process page tests/gnarly_pdfs/lots_of_sci_tables.pdf-5 cancelled 2025-07-19 23:06:34,547 - __main__ - INFO - Process page tests/gnarly_pdfs/lots_of_sci_tables.pdf-3 cancelled 2025-07-19 23:06:34,547 - __main__ - INFO - Process page tests/gnarly_pdfs/pdftotext_two_column_issue.pdf-1 cancelled 2025-07-19 23:06:34,547 - __main__ - INFO - Process page tests/gnarly_pdfs/pdftotext_two_column_issue.pdf-9 cancelled 2025-07-19 23:06:34,547 - __main__ - INFO - Process page tests/gnarly_pdfs/pdftotext_two_column_issue.pdf-14 cancelled 2025-07-19 23:06:34,547 - __main__ - INFO - Process page tests/gnarly_pdfs/pdftotext_two_column_issue.pdf-4 cancelled 2025-07-19 23:06:34,547 - __main__ - INFO - Process page tests/gnarly_pdfs/pdftotext_two_column_issue.pdf-12 cancelled 2025-07-19 23:06:34,547 - __main__ - INFO - Process page tests/gnarly_pdfs/pdftotext_two_column_issue.pdf-7 cancelled 2025-07-19 23:06:34,547 - __main__ - INFO - Process page tests/gnarly_pdfs/pdftotext_two_column_issue.pdf-2 cancelled 2025-07-19 23:06:34,547 - __main__ - INFO - Process page tests/gnarly_pdfs/pdftotext_two_column_issue.pdf-10 cancelled 2025-07-19 23:06:34,547 - __main__ - INFO - Process page tests/gnarly_pdfs/pdftotext_two_column_issue.pdf-11 cancelled 2025-07-19 23:06:34,547 - __main__ - INFO - Process page tests/gnarly_pdfs/pdftotext_two_column_issue.pdf-5 cancelled 2025-07-19 23:06:34,547 - __main__ - INFO - Process page tests/gnarly_pdfs/pdftotext_two_column_issue.pdf-13 cancelled 2025-07-19 23:06:34,548 - __main__ - INFO - Process page tests/gnarly_pdfs/pdftotext_two_column_issue.pdf-8 cancelled 2025-07-19 23:06:34,548 - __main__ - INFO - Process page tests/gnarly_pdfs/pdftotext_two_column_issue.pdf-3 cancelled 2025-07-19 23:06:34,548 - __main__ - INFO - Process page tests/gnarly_pdfs/pdftotext_two_column_issue.pdf-6 cancelled 2025-07-19 23:06:34,548 - __main__ - INFO - Process page tests/gnarly_pdfs/guidebook_failed_pages.pdf-2 cancelled 2025-07-19 23:06:34,548 - __main__ - INFO - Process page tests/gnarly_pdfs/guidebook_failed_pages.pdf-3 cancelled 2025-07-19 23:06:34,548 - __main__ - INFO - Process page tests/gnarly_pdfs/guidebook_failed_pages.pdf-1 cancelled 2025-07-19 23:06:34,548 - __main__ - INFO - Process page tests/gnarly_pdfs/large_prompt_hint3.pdf-1 cancelled 2025-07-19 23:06:34,548 - __main__ - INFO - Process page tests/gnarly_pdfs/large_prompt_hint3.pdf-3 cancelled 2025-07-19 23:06:34,548 - __main__ - INFO - Process page tests/gnarly_pdfs/large_prompt_hint3.pdf-2 cancelled 2025-07-19 23:06:34,548 - __main__ - INFO - Process page tests/gnarly_pdfs/large_prompt_hint3.pdf-4 cancelled 2025-07-19 23:06:34,548 - __main__ - INFO - Process page tests/gnarly_pdfs/large_prompt_hint1.pdf-9 cancelled 2025-07-19 23:06:34,548 - __main__ - INFO - Process page tests/gnarly_pdfs/large_prompt_hint1.pdf-19 cancelled 2025-07-19 23:06:34,633 - __main__ - INFO - Process page tests/gnarly_pdfs/large_prompt_hint1.pdf-10 cancelled 2025-07-19 23:06:34,633 - __main__ - INFO - Process page tests/gnarly_pdfs/large_prompt_hint1.pdf-20 cancelled 2025-07-19 23:06:34,633 - __main__ - INFO - Process page tests/gnarly_pdfs/large_prompt_hint1.pdf-11 cancelled 2025-07-19 23:06:34,633 - __main__ - INFO - Process page tests/gnarly_pdfs/large_prompt_hint1.pdf-21 cancelled 2025-07-19 23:06:34,633 - __main__ - INFO - Process page tests/gnarly_pdfs/large_prompt_hint1.pdf-1 cancelled 2025-07-19 23:06:34,633 - __main__ - INFO - Process page tests/gnarly_pdfs/large_prompt_hint1.pdf-12 cancelled 2025-07-19 23:06:34,633 - __main__ - INFO - Process page tests/gnarly_pdfs/large_prompt_hint1.pdf-22 cancelled 2025-07-19 23:06:34,633 - __main__ - INFO - Process page tests/gnarly_pdfs/large_prompt_hint1.pdf-2 cancelled 2025-07-19 23:06:34,634 - __main__ - INFO - Process page tests/gnarly_pdfs/large_prompt_hint1.pdf-13 cancelled 2025-07-19 23:06:34,634 - __main__ - INFO - Process page tests/gnarly_pdfs/large_prompt_hint1.pdf-23 cancelled 2025-07-19 23:06:34,634 - __main__ - INFO - Process page tests/gnarly_pdfs/large_prompt_hint1.pdf-3 cancelled 2025-07-19 23:06:34,634 - __main__ - INFO - Process page tests/gnarly_pdfs/large_prompt_hint1.pdf-14 cancelled 2025-07-19 23:06:34,634 - __main__ - INFO - Process page tests/gnarly_pdfs/large_prompt_hint1.pdf-24 cancelled 2025-07-19 23:06:34,634 - __main__ - INFO - Process page tests/gnarly_pdfs/large_prompt_hint1.pdf-4 cancelled 2025-07-19 23:06:34,634 - __main__ - INFO - Process page tests/gnarly_pdfs/large_prompt_hint1.pdf-15 cancelled 2025-07-19 23:06:34,634 - __main__ - INFO - Process page tests/gnarly_pdfs/large_prompt_hint1.pdf-25 cancelled 2025-07-19 23:06:34,634 - __main__ - INFO - Process page tests/gnarly_pdfs/large_prompt_hint1.pdf-5 cancelled 2025-07-19 23:06:34,634 - __main__ - INFO - Process page tests/gnarly_pdfs/large_prompt_hint1.pdf-16 cancelled 2025-07-19 23:06:34,634 - __main__ - INFO - Process page tests/gnarly_pdfs/large_prompt_hint1.pdf-27 cancelled 2025-07-19 23:06:34,634 - __main__ - INFO - Process page tests/gnarly_pdfs/large_prompt_hint1.pdf-6 cancelled 2025-07-19 23:06:34,634 - __main__ - INFO - Process page tests/gnarly_pdfs/large_prompt_hint1.pdf-17 cancelled 2025-07-19 23:06:34,635 - __main__ - INFO - Process page tests/gnarly_pdfs/large_prompt_hint1.pdf-7 cancelled 2025-07-19 23:06:34,635 - __main__ - INFO - Process page tests/gnarly_pdfs/large_prompt_hint1.pdf-26 cancelled 2025-07-19 23:06:34,635 - __main__ - INFO - Process page tests/gnarly_pdfs/large_prompt_hint1.pdf-8 cancelled 2025-07-19 23:06:34,635 - __main__ - INFO - Process page tests/gnarly_pdfs/large_prompt_hint1.pdf-18 cancelled 2025-07-19 23:06:34,635 - __main__ - INFO - Process page tests/gnarly_pdfs/skinnypage.pdf-2 cancelled 2025-07-19 23:06:34,635 - __main__ - INFO - Process page tests/gnarly_pdfs/skinnypage.pdf-1 cancelled 2025-07-19 23:06:34,635 - __main__ - INFO - Process page tests/gnarly_pdfs/load_v_error.pdf-7 cancelled 2025-07-19 23:06:34,635 - __main__ - INFO - Process page tests/gnarly_pdfs/load_v_error.pdf-15 cancelled 2025-07-19 23:06:34,635 - __main__ - INFO - Process page tests/gnarly_pdfs/load_v_error.pdf-12 cancelled 2025-07-19 23:06:34,635 - __main__ - INFO - Process page tests/gnarly_pdfs/load_v_error.pdf-2 cancelled 2025-07-19 23:06:34,635 - __main__ - INFO - Process page tests/gnarly_pdfs/load_v_error.pdf-10 cancelled 2025-07-19 23:06:34,635 - __main__ - INFO - Process page tests/gnarly_pdfs/load_v_error.pdf-5 cancelled 2025-07-19 23:06:34,636 - __main__ - INFO - Process page tests/gnarly_pdfs/load_v_error.pdf-13 cancelled 2025-07-19 23:06:34,636 - __main__ - INFO - Process page tests/gnarly_pdfs/load_v_error.pdf-8 cancelled 2025-07-19 23:06:34,636 - __main__ - INFO - Process page tests/gnarly_pdfs/load_v_error.pdf-9 cancelled 2025-07-19 23:06:34,636 - __main__ - INFO - Process page tests/gnarly_pdfs/load_v_error.pdf-16 cancelled 2025-07-19 23:06:34,636 - __main__ - INFO - Process page tests/gnarly_pdfs/load_v_error.pdf-3 cancelled 2025-07-19 23:06:34,636 - __main__ - INFO - Process page tests/gnarly_pdfs/load_v_error.pdf-11 cancelled 2025-07-19 23:06:34,636 - __main__ - INFO - Process page tests/gnarly_pdfs/load_v_error.pdf-6 cancelled 2025-07-19 23:06:34,636 - __main__ - INFO - Process page tests/gnarly_pdfs/load_v_error.pdf-14 cancelled 2025-07-19 23:06:34,636 - __main__ - INFO - Process page tests/gnarly_pdfs/load_v_error.pdf-1 cancelled 2025-07-19 23:06:34,636 - __main__ - INFO - Process page tests/gnarly_pdfs/load_v_error.pdf-4 cancelled 2025-07-19 23:06:34,636 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_good_some_pages_should_get_filtered.pdf-13 cancelled 2025-07-19 23:06:34,637 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_good_some_pages_should_get_filtered.pdf-23 cancelled 2025-07-19 23:06:34,637 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_good_some_pages_should_get_filtered.pdf-3 cancelled 2025-07-19 23:06:34,637 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_good_some_pages_should_get_filtered.pdf-14 cancelled 2025-07-19 23:06:34,637 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_good_some_pages_should_get_filtered.pdf-24 cancelled 2025-07-19 23:06:34,637 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_good_some_pages_should_get_filtered.pdf-4 cancelled 2025-07-19 23:06:34,637 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_good_some_pages_should_get_filtered.pdf-15 cancelled 2025-07-19 23:06:34,637 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_good_some_pages_should_get_filtered.pdf-25 cancelled 2025-07-19 23:06:34,637 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_good_some_pages_should_get_filtered.pdf-5 cancelled 2025-07-19 23:06:34,637 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_good_some_pages_should_get_filtered.pdf-16 cancelled 2025-07-19 23:06:34,637 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_good_some_pages_should_get_filtered.pdf-6 cancelled 2025-07-19 23:06:34,637 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_good_some_pages_should_get_filtered.pdf-17 cancelled 2025-07-19 23:06:34,637 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_good_some_pages_should_get_filtered.pdf-7 cancelled 2025-07-19 23:06:34,637 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_good_some_pages_should_get_filtered.pdf-26 cancelled 2025-07-19 23:06:34,638 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_good_some_pages_should_get_filtered.pdf-8 cancelled 2025-07-19 23:06:34,638 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_good_some_pages_should_get_filtered.pdf-18 cancelled 2025-07-19 23:06:34,638 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_good_some_pages_should_get_filtered.pdf-9 cancelled 2025-07-19 23:06:34,638 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_good_some_pages_should_get_filtered.pdf-19 cancelled 2025-07-19 23:06:34,638 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_good_some_pages_should_get_filtered.pdf-10 cancelled 2025-07-19 23:06:34,638 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_good_some_pages_should_get_filtered.pdf-20 cancelled 2025-07-19 23:06:34,638 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_good_some_pages_should_get_filtered.pdf-11 cancelled 2025-07-19 23:06:34,638 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_good_some_pages_should_get_filtered.pdf-21 cancelled 2025-07-19 23:06:34,638 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_good_some_pages_should_get_filtered.pdf-1 cancelled 2025-07-19 23:06:34,638 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_good_some_pages_should_get_filtered.pdf-12 cancelled 2025-07-19 23:06:34,638 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_good_some_pages_should_get_filtered.pdf-22 cancelled 2025-07-19 23:06:34,638 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_good_some_pages_should_get_filtered.pdf-2 cancelled 2025-07-19 23:06:34,638 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-4 cancelled 2025-07-19 23:06:34,638 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-89 cancelled 2025-07-19 23:06:34,639 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-47 cancelled 2025-07-19 23:06:34,639 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-5 cancelled 2025-07-19 23:06:34,639 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-90 cancelled 2025-07-19 23:06:34,639 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-48 cancelled 2025-07-19 23:06:34,639 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-6 cancelled 2025-07-19 23:06:34,639 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-91 cancelled 2025-07-19 23:06:34,639 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-49 cancelled 2025-07-19 23:06:34,639 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-7 cancelled 2025-07-19 23:06:34,639 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-92 cancelled 2025-07-19 23:06:34,639 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-50 cancelled 2025-07-19 23:06:34,639 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-8 cancelled 2025-07-19 23:06:34,639 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-93 cancelled 2025-07-19 23:06:34,639 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-51 cancelled 2025-07-19 23:06:34,639 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-9 cancelled 2025-07-19 23:06:34,640 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-94 cancelled 2025-07-19 23:06:34,640 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-52 cancelled 2025-07-19 23:06:34,640 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-10 cancelled 2025-07-19 23:06:34,640 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-95 cancelled 2025-07-19 23:06:34,640 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-53 cancelled 2025-07-19 23:06:34,640 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-11 cancelled 2025-07-19 23:06:34,640 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-96 cancelled 2025-07-19 23:06:34,640 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-54 cancelled 2025-07-19 23:06:34,640 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-12 cancelled 2025-07-19 23:06:34,640 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-97 cancelled 2025-07-19 23:06:34,641 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-55 cancelled 2025-07-19 23:06:34,641 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-13 cancelled 2025-07-19 23:06:34,641 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-98 cancelled 2025-07-19 23:06:34,641 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-56 cancelled 2025-07-19 23:06:34,641 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-14 cancelled 2025-07-19 23:06:34,641 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-99 cancelled 2025-07-19 23:06:34,641 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-57 cancelled 2025-07-19 23:06:34,641 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-15 cancelled 2025-07-19 23:06:34,641 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-100 cancelled 2025-07-19 23:06:34,641 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-58 cancelled 2025-07-19 23:06:34,641 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-16 cancelled 2025-07-19 23:06:34,641 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-101 cancelled 2025-07-19 23:06:34,642 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-59 cancelled 2025-07-19 23:06:34,642 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-17 cancelled 2025-07-19 23:06:34,642 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-102 cancelled 2025-07-19 23:06:34,642 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-60 cancelled 2025-07-19 23:06:34,642 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-26 cancelled 2025-07-19 23:06:34,642 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-103 cancelled 2025-07-19 23:06:34,642 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-61 cancelled 2025-07-19 23:06:34,642 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-18 cancelled 2025-07-19 23:06:34,642 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-104 cancelled 2025-07-19 23:06:34,642 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-62 cancelled 2025-07-19 23:06:34,642 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-19 cancelled 2025-07-19 23:06:34,642 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-105 cancelled 2025-07-19 23:06:34,643 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-63 cancelled 2025-07-19 23:06:34,643 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-20 cancelled 2025-07-19 23:06:34,643 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-106 cancelled 2025-07-19 23:06:34,643 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-64 cancelled 2025-07-19 23:06:34,643 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-21 cancelled 2025-07-19 23:06:34,643 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-65 cancelled 2025-07-19 23:06:34,643 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-22 cancelled 2025-07-19 23:06:34,643 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-66 cancelled 2025-07-19 23:06:34,643 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-23 cancelled 2025-07-19 23:06:34,643 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-67 cancelled 2025-07-19 23:06:34,644 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-24 cancelled 2025-07-19 23:06:34,644 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-68 cancelled 2025-07-19 23:06:34,644 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-25 cancelled 2025-07-19 23:06:34,644 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-69 cancelled 2025-07-19 23:06:34,644 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-27 cancelled 2025-07-19 23:06:34,644 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-70 cancelled 2025-07-19 23:06:34,644 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-28 cancelled 2025-07-19 23:06:34,644 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-71 cancelled 2025-07-19 23:06:34,644 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-29 cancelled 2025-07-19 23:06:34,644 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-72 cancelled 2025-07-19 23:06:34,644 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-30 cancelled 2025-07-19 23:06:34,644 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-73 cancelled 2025-07-19 23:06:34,644 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-31 cancelled 2025-07-19 23:06:34,644 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-74 cancelled 2025-07-19 23:06:34,644 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-32 cancelled 2025-07-19 23:06:34,644 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-75 cancelled 2025-07-19 23:06:34,645 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-33 cancelled 2025-07-19 23:06:34,645 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-76 cancelled 2025-07-19 23:06:34,645 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-34 cancelled 2025-07-19 23:06:34,645 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-77 cancelled 2025-07-19 23:06:34,645 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-35 cancelled 2025-07-19 23:06:34,645 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-78 cancelled 2025-07-19 23:06:34,645 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-36 cancelled 2025-07-19 23:06:34,645 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-79 cancelled 2025-07-19 23:06:34,645 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-37 cancelled 2025-07-19 23:06:34,645 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-80 cancelled 2025-07-19 23:06:34,645 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-38 cancelled 2025-07-19 23:06:34,645 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-81 cancelled 2025-07-19 23:06:34,645 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-39 cancelled 2025-07-19 23:06:34,645 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-82 cancelled 2025-07-19 23:06:34,645 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-40 cancelled 2025-07-19 23:06:34,645 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-83 cancelled 2025-07-19 23:06:34,645 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-41 cancelled 2025-07-19 23:06:34,645 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-84 cancelled 2025-07-19 23:06:34,645 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-42 cancelled 2025-07-19 23:06:34,645 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-85 cancelled 2025-07-19 23:06:34,645 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-43 cancelled 2025-07-19 23:06:34,645 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-1 cancelled 2025-07-19 23:06:34,645 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-86 cancelled 2025-07-19 23:06:34,645 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-44 cancelled 2025-07-19 23:06:34,645 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-2 cancelled 2025-07-19 23:06:34,646 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-87 cancelled 2025-07-19 23:06:34,646 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-45 cancelled 2025-07-19 23:06:34,646 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-3 cancelled 2025-07-19 23:06:34,646 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-88 cancelled 2025-07-19 23:06:34,646 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-46 cancelled 2025-07-19 23:06:34,646 - __main__ - INFO - Process page tests/gnarly_pdfs/repeating_references_on_pg9_pg10.pdf-2 cancelled 2025-07-19 23:06:34,646 - __main__ - INFO - Process page tests/gnarly_pdfs/repeating_references_on_pg9_pg10.pdf-10 cancelled 2025-07-19 23:06:34,646 - __main__ - INFO - Process page tests/gnarly_pdfs/repeating_references_on_pg9_pg10.pdf-5 cancelled 2025-07-19 23:06:34,646 - __main__ - INFO - Process page tests/gnarly_pdfs/repeating_references_on_pg9_pg10.pdf-8 cancelled 2025-07-19 23:06:34,646 - __main__ - INFO - Process page tests/gnarly_pdfs/repeating_references_on_pg9_pg10.pdf-3 cancelled 2025-07-19 23:06:34,646 - __main__ - INFO - Process page tests/gnarly_pdfs/repeating_references_on_pg9_pg10.pdf-6 cancelled 2025-07-19 23:06:34,646 - __main__ - INFO - Process page tests/gnarly_pdfs/repeating_references_on_pg9_pg10.pdf-1 cancelled 2025-07-19 23:06:34,646 - __main__ - INFO - Process page tests/gnarly_pdfs/repeating_references_on_pg9_pg10.pdf-9 cancelled 2025-07-19 23:06:34,646 - __main__ - INFO - Process page tests/gnarly_pdfs/repeating_references_on_pg9_pg10.pdf-4 cancelled 2025-07-19 23:06:34,646 - __main__ - INFO - Process page tests/gnarly_pdfs/repeating_references_on_pg9_pg10.pdf-7 cancelled 2025-07-19 23:06:34,646 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_images.pdf-32 cancelled 2025-07-19 23:06:34,646 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_images.pdf-11 cancelled 2025-07-19 23:06:34,646 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_images.pdf-43 cancelled 2025-07-19 23:06:34,646 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_images.pdf-64 cancelled 2025-07-19 23:06:34,646 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_images.pdf-21 cancelled 2025-07-19 23:06:34,646 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_images.pdf-53 cancelled 2025-07-19 23:06:34,646 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_images.pdf-54 cancelled 2025-07-19 23:06:34,646 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_images.pdf-1 cancelled 2025-07-19 23:06:34,646 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_images.pdf-33 cancelled 2025-07-19 23:06:34,646 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_images.pdf-12 cancelled 2025-07-19 23:06:34,647 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_images.pdf-44 cancelled 2025-07-19 23:06:34,647 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_images.pdf-65 cancelled 2025-07-19 23:06:34,647 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_images.pdf-22 cancelled 2025-07-19 23:06:34,647 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_images.pdf-55 cancelled 2025-07-19 23:06:34,647 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_images.pdf-2 cancelled 2025-07-19 23:06:34,647 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_images.pdf-34 cancelled 2025-07-19 23:06:34,647 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_images.pdf-13 cancelled 2025-07-19 23:06:34,647 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_images.pdf-45 cancelled 2025-07-19 23:06:34,647 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_images.pdf-66 cancelled 2025-07-19 23:06:34,647 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_images.pdf-23 cancelled 2025-07-19 23:06:34,647 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_images.pdf-56 cancelled 2025-07-19 23:06:34,647 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_images.pdf-3 cancelled 2025-07-19 23:06:34,647 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_images.pdf-35 cancelled 2025-07-19 23:06:34,647 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_images.pdf-14 cancelled 2025-07-19 23:06:34,647 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_images.pdf-46 cancelled 2025-07-19 23:06:34,647 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_images.pdf-67 cancelled 2025-07-19 23:06:34,647 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_images.pdf-24 cancelled 2025-07-19 23:06:34,647 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_images.pdf-57 cancelled 2025-07-19 23:06:34,647 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_images.pdf-4 cancelled 2025-07-19 23:06:34,647 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_images.pdf-36 cancelled 2025-07-19 23:06:34,647 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_images.pdf-15 cancelled 2025-07-19 23:06:34,647 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_images.pdf-47 cancelled 2025-07-19 23:06:34,647 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_images.pdf-68 cancelled 2025-07-19 23:06:34,647 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_images.pdf-25 cancelled 2025-07-19 23:06:34,647 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_images.pdf-58 cancelled 2025-07-19 23:06:34,648 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_images.pdf-5 cancelled 2025-07-19 23:06:34,648 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_images.pdf-37 cancelled 2025-07-19 23:06:34,648 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_images.pdf-16 cancelled 2025-07-19 23:06:34,648 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_images.pdf-48 cancelled 2025-07-19 23:06:34,648 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_images.pdf-27 cancelled 2025-07-19 23:06:34,648 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_images.pdf-59 cancelled 2025-07-19 23:06:34,648 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_images.pdf-6 cancelled 2025-07-19 23:06:34,648 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_images.pdf-38 cancelled 2025-07-19 23:06:34,648 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_images.pdf-17 cancelled 2025-07-19 23:06:34,648 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_images.pdf-49 cancelled 2025-07-19 23:06:34,648 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_images.pdf-28 cancelled 2025-07-19 23:06:34,648 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_images.pdf-60 cancelled 2025-07-19 23:06:34,648 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_images.pdf-7 cancelled 2025-07-19 23:06:34,648 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_images.pdf-39 cancelled 2025-07-19 23:06:34,648 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_images.pdf-26 cancelled 2025-07-19 23:06:34,648 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_images.pdf-50 cancelled 2025-07-19 23:06:34,648 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_images.pdf-29 cancelled 2025-07-19 23:06:34,648 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_images.pdf-61 cancelled 2025-07-19 23:06:34,648 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_images.pdf-8 cancelled 2025-07-19 23:06:34,648 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_images.pdf-40 cancelled 2025-07-19 23:06:34,648 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_images.pdf-18 cancelled 2025-07-19 23:06:34,648 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_images.pdf-51 cancelled 2025-07-19 23:06:34,648 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_images.pdf-30 cancelled 2025-07-19 23:06:34,648 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_images.pdf-62 cancelled 2025-07-19 23:06:34,648 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_images.pdf-9 cancelled 2025-07-19 23:06:34,648 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_images.pdf-41 cancelled 2025-07-19 23:06:34,649 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_images.pdf-19 cancelled 2025-07-19 23:06:34,649 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_images.pdf-42 cancelled 2025-07-19 23:06:34,649 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_images.pdf-52 cancelled 2025-07-19 23:06:34,649 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_images.pdf-31 cancelled 2025-07-19 23:06:34,649 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_images.pdf-63 cancelled 2025-07-19 23:06:34,649 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_images.pdf-10 cancelled 2025-07-19 23:06:34,649 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_images.pdf-20 cancelled 2025-07-19 23:06:34,649 - __main__ - INFO - Process page tests/gnarly_pdfs/form_on_later_pages.pdf-7 cancelled 2025-07-19 23:06:34,649 - __main__ - INFO - Process page tests/gnarly_pdfs/form_on_later_pages.pdf-2 cancelled 2025-07-19 23:06:34,649 - __main__ - INFO - Process page tests/gnarly_pdfs/form_on_later_pages.pdf-10 cancelled 2025-07-19 23:06:34,649 - __main__ - INFO - Process page tests/gnarly_pdfs/form_on_later_pages.pdf-5 cancelled 2025-07-19 23:06:34,649 - __main__ - INFO - Process page tests/gnarly_pdfs/form_on_later_pages.pdf-8 cancelled 2025-07-19 23:06:34,649 - __main__ - INFO - Process page tests/gnarly_pdfs/form_on_later_pages.pdf-3 cancelled 2025-07-19 23:06:34,649 - __main__ - INFO - Process page tests/gnarly_pdfs/form_on_later_pages.pdf-6 cancelled 2025-07-19 23:06:34,649 - __main__ - INFO - Process page tests/gnarly_pdfs/form_on_later_pages.pdf-9 cancelled 2025-07-19 23:06:34,649 - __main__ - INFO - Process page tests/gnarly_pdfs/form_on_later_pages.pdf-1 cancelled 2025-07-19 23:06:34,649 - __main__ - INFO - Process page tests/gnarly_pdfs/form_on_later_pages.pdf-4 cancelled 2025-07-19 23:06:34,649 - __main__ - INFO - Process page tests/gnarly_pdfs/handwriting_bad_ocr.pdf-1 cancelled 2025-07-19 23:06:34,649 - __main__ - INFO - Process page tests/gnarly_pdfs/handwriting_bad_ocr.pdf-2 cancelled 2025-07-19 23:06:34,649 - __main__ - INFO - Process page tests/gnarly_pdfs/map1.pdf-1 cancelled 2025-07-19 23:06:34,649 - __main__ - INFO - Process page tests/gnarly_pdfs/small_page_size.pdf-1 cancelled 2025-07-19 23:06:34,649 - __main__ - INFO - Process page tests/gnarly_pdfs/most_content_in_image_form.pdf-3 cancelled 2025-07-19 23:06:34,649 - __main__ - INFO - Process page tests/gnarly_pdfs/most_content_in_image_form.pdf-6 cancelled 2025-07-19 23:06:34,649 - __main__ - INFO - Process page tests/gnarly_pdfs/most_content_in_image_form.pdf-1 cancelled 2025-07-19 23:06:34,649 - __main__ - INFO - Process page tests/gnarly_pdfs/most_content_in_image_form.pdf-4 cancelled 2025-07-19 23:06:34,649 - __main__ - INFO - Process page tests/gnarly_pdfs/most_content_in_image_form.pdf-7 cancelled 2025-07-19 23:06:34,650 - __main__ - INFO - Process page tests/gnarly_pdfs/most_content_in_image_form.pdf-2 cancelled 2025-07-19 23:06:34,650 - __main__ - INFO - Process page tests/gnarly_pdfs/most_content_in_image_form.pdf-5 cancelled 2025-07-19 23:06:34,650 - __main__ - INFO - Process page tests/gnarly_pdfs/some_ocr1.pdf-1 cancelled 2025-07-19 23:06:34,650 - __main__ - INFO - Process page tests/gnarly_pdfs/newspaper.pdf-1 cancelled 2025-07-19 23:06:34,650 - __main__ - INFO - Got cancellation request for SGLang server 2025-07-19 23:07:14,182 - __main__ - INFO - Got --pdfs argument, going to add to the work queue 2025-07-19 23:07:14,182 - __main__ - INFO - Loading file at scripts/data/11440000MB2D0234372440125017009.pdf as PDF document 2025-07-19 23:07:14,182 - __main__ - INFO - Loading file at scripts/data/11440000MB2D0234372440125017014.pdf as PDF document 2025-07-19 23:07:14,182 - __main__ - INFO - Loading file at scripts/data/11440000MB2D0234372440125017020.pdf as PDF document 2025-07-19 23:07:14,183 - __main__ - INFO - Loading file at scripts/data/11440000MB2D0234372440125017028.pdf as PDF document 2025-07-19 23:07:14,183 - __main__ - INFO - Loading file at scripts/data/11440000MB2D0234372440125017041.pdf as PDF document 2025-07-19 23:07:14,183 - __main__ - INFO - Loading file at scripts/data/11440000MB2D0234372440125017049.pdf as PDF document 2025-07-19 23:07:14,183 - __main__ - INFO - Loading file at scripts/data/11445200MB2C47380T4440125017008 (1).pdf as PDF document 2025-07-19 23:07:14,183 - __main__ - INFO - Loading file at scripts/data/11445200MB2C47380T4440125017008.pdf as PDF document 2025-07-19 23:07:14,183 - __main__ - INFO - Loading file at scripts/data/11445200MB2C47380T4440125017023.pdf as PDF document 2025-07-19 23:07:14,183 - __main__ - INFO - Loading file at scripts/data/11445200MB2D06387W3440125011001.pdf as PDF document 2025-07-19 23:07:14,183 - __main__ - INFO - Loading file at scripts/data/11445200MB2D06387W3440125017003.pdf as PDF document 2025-07-19 23:07:14,183 - __main__ - INFO - Loading file at scripts/data/11445200MB2D06387W3440125017006.pdf as PDF document 2025-07-19 23:07:14,183 - __main__ - INFO - Loading file at scripts/data/11445200MB2D06387W3440125017007.pdf as PDF document 2025-07-19 23:07:14,183 - __main__ - INFO - Loading file at scripts/data/11445200MB2D06387W3440125017011.pdf as PDF document 2025-07-19 23:07:14,184 - __main__ - INFO - Loading file at scripts/data/11445200MB2D06387W3440125017023.pdf as PDF document 2025-07-19 23:07:14,184 - __main__ - INFO - Loading file at scripts/data/11445200MB2D06387W3440125017041.pdf as PDF document 2025-07-19 23:07:14,184 - __main__ - INFO - Loading file at scripts/data/11445200MB2D06387W3440125017048.pdf as PDF document 2025-07-19 23:07:14,184 - __main__ - INFO - Loading file at scripts/data/11445200MB2D42580L4442014010000.pdf as PDF document 2025-07-19 23:07:14,184 - __main__ - INFO - Loading file at scripts/data/11445200MB2D6222364440125017008.pdf as PDF document 2025-07-19 23:07:14,184 - __main__ - INFO - Loading file at scripts/data/11445200MB2D6222364440125017049.pdf as PDF document 2025-07-19 23:07:14,184 - __main__ - INFO - Loading file at scripts/data/11445202592174409C4442111641000.pdf as PDF document 2025-07-19 23:07:14,185 - __main__ - INFO - Loading file at scripts/data/11445202592174409C4442111667001.pdf as PDF document 2025-07-19 23:07:14,185 - __main__ - INFO - Loading file at scripts/data/11445202592174409C4442111820005.pdf as PDF document 2025-07-19 23:07:14,185 - __main__ - INFO - Loading file at scripts/data/11445202MB2D1177604440125017023.pdf as PDF document 2025-07-19 23:07:14,185 - __main__ - INFO - Loading file at scripts/data/11445202MB2D1177604440125017027.pdf as PDF document 2025-07-19 23:07:14,185 - __main__ - INFO - Loading file at scripts/data/11445202MB2D1177604440125017041.pdf as PDF document 2025-07-19 23:07:14,185 - __main__ - INFO - Loading file at scripts/data/11445202MB2D117760444212503R001.pdf as PDF document 2025-07-19 23:07:14,185 - __main__ - INFO - Loading file at scripts/data/11445203007030456U4440711000000.pdf as PDF document 2025-07-19 23:07:14,185 - __main__ - INFO - Loading file at scripts/data/11445203007030456U44421110A0005.pdf as PDF document 2025-07-19 23:07:14,185 - __main__ - INFO - Loading file at scripts/data/11445203007030456U4442111640000.pdf as PDF document 2025-07-19 23:07:14,185 - __main__ - INFO - Loading file at scripts/data/11445203007030456U4442111641000.pdf as PDF document 2025-07-19 23:07:14,185 - __main__ - INFO - Loading file at scripts/data/11445203007030456U4442111667001.pdf as PDF document 2025-07-19 23:07:14,185 - __main__ - INFO - Loading file at scripts/data/11445203707759010G4442014010000.pdf as PDF document 2025-07-19 23:07:14,186 - __main__ - INFO - Loading file at scripts/data/11445203MB2C21084N4440125017008.pdf as PDF document 2025-07-19 23:07:14,186 - __main__ - INFO - Loading file at scripts/data/11445203MB2C21084N444212503R001.pdf as PDF document 2025-07-19 23:07:14,186 - __main__ - INFO - Loading file at scripts/data/11445222007029500K4440711000000.pdf as PDF document 2025-07-19 23:07:14,186 - __main__ - INFO - Loading file at scripts/data/11445222007029500K44421110A0001.pdf as PDF document 2025-07-19 23:07:14,186 - __main__ - INFO - Loading file at scripts/data/11445222007029500K44421110A0005.pdf as PDF document 2025-07-19 23:07:14,186 - __main__ - INFO - Loading file at scripts/data/11445222007029527B4442106100010.pdf as PDF document 2025-07-19 23:07:14,186 - __main__ - INFO - Loading file at scripts/data/11445222007030157E4440149001001.pdf as PDF document 2025-07-19 23:07:14,186 - __main__ - INFO - Loading file at scripts/data/11445224007035644H4440711000000.pdf as PDF document 2025-07-19 23:07:14,186 - __main__ - INFO - Loading file at scripts/data/11445224007035644H44421110A0001.pdf as PDF document 2025-07-19 23:07:14,186 - __main__ - INFO - Loading file at scripts/data/11445224007035644H44421110A0005.pdf as PDF document 2025-07-19 23:07:14,186 - __main__ - INFO - Loading file at scripts/data/11445224007035652C4440114020001.pdf as PDF document 2025-07-19 23:07:14,187 - __main__ - INFO - Loading file at scripts/data/11445224007035652C4442014010000.pdf as PDF document 2025-07-19 23:07:14,187 - __main__ - INFO - Loading file at scripts/data/11445281588281455A4440711000000.pdf as PDF document 2025-07-19 23:07:14,187 - __main__ - INFO - Loading file at scripts/data/11445281588281455A44421110A0001.pdf as PDF document 2025-07-19 23:07:14,187 - __main__ - INFO - Loading file at scripts/data/11445281588281455A44421110A0005.pdf as PDF document 2025-07-19 23:07:14,187 - __main__ - INFO - Loading file at scripts/data/11445281588281455A4442111641000.pdf as PDF document 2025-07-19 23:07:14,187 - __main__ - INFO - Loading file at scripts/data/11445281588281455A4442111667001.pdf as PDF document 2025-07-19 23:07:14,187 - __main__ - INFO - Loading file at scripts/data/11445281588281455A4442111820005.pdf as PDF document 2025-07-19 23:07:14,187 - __main__ - INFO - Loading file at scripts/data/12445200456019383L3442111667001.pdf as PDF document 2025-07-19 23:07:14,187 - __main__ - INFO - Loading file at scripts/data/12445200726503846U344201405500301.pdf as PDF document 2025-07-19 23:07:14,187 - __main__ - INFO - Loading file at scripts/data/12445200726503846U3442014055009.pdf as PDF document 2025-07-19 23:07:14,187 - __main__ - INFO - Found 54 total pdf paths to add 2025-07-19 23:07:14,306 - __main__ - INFO - Calculated items_per_group: 53 based on average pages per PDF: 9.35 2025-07-19 23:07:14,512 - __main__ - INFO - Starting pipeline with PID 555339 2025-07-19 23:07:14,512 - __main__ - INFO - Downloading model with hugging face 'allenai/olmOCR-7B-0225-preview' 2025-07-19 23:07:25,124 - __main__ - WARNING - Attempt 1: Please wait for sglang server to become ready... 2025-07-19 23:07:27,569 - sglang - INFO - [2025-07-19 23:07:27] server_args=ServerArgs(model_path='allenai/olmOCR-7B-0225-preview', tokenizer_path='allenai/olmOCR-7B-0225-preview', tokenizer_mode='auto', load_format='auto', trust_remote_code=False, dtype='auto', kv_cache_dtype='auto', quantization_param_path=None, quantization=None, context_length=None, device='cuda', served_model_name='allenai/olmOCR-7B-0225-preview', chat_template='qwen2-vl', is_embedding=False, revision=None, skip_tokenizer_init=False, host='127.0.0.1', port=30024, mem_fraction_static=0.8, max_running_requests=None, max_total_tokens=None, chunked_prefill_size=2048, max_prefill_tokens=16384, schedule_policy='lpm', schedule_conservativeness=1.0, cpu_offload_gb=0, prefill_only_one_req=False, tp_size=1, stream_interval=1, stream_output=False, random_seed=495738545, constrained_json_whitespace_pattern=None, watchdog_timeout=300, download_dir=None, base_gpu_id=0, log_level='info', log_level_http='warning', log_requests=False, show_time_cost=False, enable_metrics=False, decode_log_interval=40, api_key=None, file_storage_pth='sglang_storage', enable_cache_report=False, dp_size=1, load_balance_method='round_robin', ep_size=1, dist_init_addr=None, nnodes=1, node_rank=0, json_model_override_args='{}', lora_paths=None, max_loras_per_batch=8, attention_backend='flashinfer', sampling_backend='flashinfer', grammar_backend='outlines', speculative_draft_model_path=None, speculative_algorithm=None, speculative_num_steps=5, speculative_num_draft_tokens=64, speculative_eagle_topk=8, enable_double_sparsity=False, ds_channel_config_path=None, ds_heavy_channel_num=32, ds_heavy_token_num=256, ds_heavy_channel_type='qk', ds_sparse_decode_threshold=4096, disable_radix_cache=False, disable_jump_forward=False, disable_cuda_graph=False, disable_cuda_graph_padding=False, disable_outlines_disk_cache=False, disable_custom_all_reduce=False, disable_mla=False, disable_overlap_schedule=False, enable_mixed_chunk=False, enable_dp_attention=False, enable_ep_moe=False, enable_torch_compile=False, torch_compile_max_bs=32, cuda_graph_max_bs=8, cuda_graph_bs=None, torchao_config='', enable_nan_detection=False, enable_p2p_check=False, triton_attention_reduce_in_fp32=False, triton_attention_num_kv_splits=8, num_continuous_decode_steps=1, delete_ckpt_after_loading=False, enable_memory_saver=False, allow_auto_truncate=False, enable_custom_logit_processor=False, tool_call_parser=None) 2025-07-19 23:07:27,569 - __main__ - INFO - [2025-07-19 23:07:27] server_args=ServerArgs(model_path='allenai/olmOCR-7B-0225-preview', tokenizer_path='allenai/olmOCR-7B-0225-preview', tokenizer_mode='auto', load_format='auto', trust_remote_code=False, dtype='auto', kv_cache_dtype='auto', quantization_param_path=None, quantization=None, context_length=None, device='cuda', served_model_name='allenai/olmOCR-7B-0225-preview', chat_template='qwen2-vl', is_embedding=False, revision=None, skip_tokenizer_init=False, host='127.0.0.1', port=30024, mem_fraction_static=0.8, max_running_requests=None, max_total_tokens=None, chunked_prefill_size=2048, max_prefill_tokens=16384, schedule_policy='lpm', schedule_conservativeness=1.0, cpu_offload_gb=0, prefill_only_one_req=False, tp_size=1, stream_interval=1, stream_output=False, random_seed=495738545, constrained_json_whitespace_pattern=None, watchdog_timeout=300, download_dir=None, base_gpu_id=0, log_level='info', log_level_http='warning', log_requests=False, show_time_cost=False, enable_metrics=False, decode_log_interval=40, api_key=None, file_storage_pth='sglang_storage', enable_cache_report=False, dp_size=1, load_balance_method='round_robin', ep_size=1, dist_init_addr=None, nnodes=1, node_rank=0, json_model_override_args='{}', lora_paths=None, max_loras_per_batch=8, attention_backend='flashinfer', sampling_backend='flashinfer', grammar_backend='outlines', speculative_draft_model_path=None, speculative_algorithm=None, speculative_num_steps=5, speculative_num_draft_tokens=64, speculative_eagle_topk=8, enable_double_sparsity=False, ds_channel_config_path=None, ds_heavy_channel_num=32, ds_heavy_token_num=256, ds_heavy_channel_type='qk', ds_sparse_decode_threshold=4096, disable_radix_cache=False, disable_jump_forward=False, disable_cuda_graph=False, disable_cuda_graph_padding=False, disable_outlines_disk_cache=False, disable_custom_all_reduce=False, disable_mla=False, disable_overlap_schedule=False, enable_mixed_chunk=False, enable_dp_attention=False, enable_ep_moe=False, enable_torch_compile=False, torch_compile_max_bs=32, cuda_graph_max_bs=8, cuda_graph_bs=None, torchao_config='', enable_nan_detection=False, enable_p2p_check=False, triton_attention_reduce_in_fp32=False, triton_attention_num_kv_splits=8, num_continuous_decode_steps=1, delete_ckpt_after_loading=False, enable_memory_saver=False, allow_auto_truncate=False, enable_custom_logit_processor=False, tool_call_parser=None) 2025-07-19 23:07:31,183 - __main__ - WARNING - Attempt 2: Please wait for sglang server to become ready... 2025-07-19 23:07:33,574 - sglang - INFO - [2025-07-19 23:07:33] Use chat template for the OpenAI-compatible API server: qwen2-vl 2025-07-19 23:07:33,574 - __main__ - INFO - [2025-07-19 23:07:33] Use chat template for the OpenAI-compatible API server: qwen2-vl 2025-07-19 23:07:37,266 - __main__ - WARNING - Attempt 3: Please wait for sglang server to become ready... 2025-07-19 23:07:43,350 - __main__ - WARNING - Attempt 4: Please wait for sglang server to become ready... 2025-07-19 23:07:43,353 - __main__ - INFO - Got cancellation request for SGLang server 2025-07-19 23:08:05,369 - __main__ - INFO - Got --pdfs argument, going to add to the work queue 2025-07-19 23:08:05,369 - __main__ - INFO - Loading file at scripts/data/11440000MB2D0234372440125017014.pdf as PDF document 2025-07-19 23:08:05,369 - __main__ - INFO - Found 1 total pdf paths to add 2025-07-19 23:08:05,374 - __main__ - INFO - Calculated items_per_group: 27 based on average pages per PDF: 18.00 2025-07-19 23:08:05,594 - __main__ - INFO - Starting pipeline with PID 556062 2025-07-19 23:08:05,594 - __main__ - INFO - Downloading model with hugging face 'allenai/olmOCR-7B-0225-preview' 2025-07-19 23:08:11,182 - __main__ - WARNING - Attempt 1: Please wait for sglang server to become ready... 2025-07-19 23:08:13,754 - sglang - INFO - [2025-07-19 23:08:13] server_args=ServerArgs(model_path='allenai/olmOCR-7B-0225-preview', tokenizer_path='allenai/olmOCR-7B-0225-preview', tokenizer_mode='auto', load_format='auto', trust_remote_code=False, dtype='auto', kv_cache_dtype='auto', quantization_param_path=None, quantization=None, context_length=None, device='cuda', served_model_name='allenai/olmOCR-7B-0225-preview', chat_template='qwen2-vl', is_embedding=False, revision=None, skip_tokenizer_init=False, host='127.0.0.1', port=30024, mem_fraction_static=0.8, max_running_requests=None, max_total_tokens=None, chunked_prefill_size=2048, max_prefill_tokens=16384, schedule_policy='lpm', schedule_conservativeness=1.0, cpu_offload_gb=0, prefill_only_one_req=False, tp_size=1, stream_interval=1, stream_output=False, random_seed=266199639, constrained_json_whitespace_pattern=None, watchdog_timeout=300, download_dir=None, base_gpu_id=0, log_level='info', log_level_http='warning', log_requests=False, show_time_cost=False, enable_metrics=False, decode_log_interval=40, api_key=None, file_storage_pth='sglang_storage', enable_cache_report=False, dp_size=1, load_balance_method='round_robin', ep_size=1, dist_init_addr=None, nnodes=1, node_rank=0, json_model_override_args='{}', lora_paths=None, max_loras_per_batch=8, attention_backend='flashinfer', sampling_backend='flashinfer', grammar_backend='outlines', speculative_draft_model_path=None, speculative_algorithm=None, speculative_num_steps=5, speculative_num_draft_tokens=64, speculative_eagle_topk=8, enable_double_sparsity=False, ds_channel_config_path=None, ds_heavy_channel_num=32, ds_heavy_token_num=256, ds_heavy_channel_type='qk', ds_sparse_decode_threshold=4096, disable_radix_cache=False, disable_jump_forward=False, disable_cuda_graph=False, disable_cuda_graph_padding=False, disable_outlines_disk_cache=False, disable_custom_all_reduce=False, disable_mla=False, disable_overlap_schedule=False, enable_mixed_chunk=False, enable_dp_attention=False, enable_ep_moe=False, enable_torch_compile=False, torch_compile_max_bs=32, cuda_graph_max_bs=8, cuda_graph_bs=None, torchao_config='', enable_nan_detection=False, enable_p2p_check=False, triton_attention_reduce_in_fp32=False, triton_attention_num_kv_splits=8, num_continuous_decode_steps=1, delete_ckpt_after_loading=False, enable_memory_saver=False, allow_auto_truncate=False, enable_custom_logit_processor=False, tool_call_parser=None) 2025-07-19 23:08:13,754 - __main__ - INFO - [2025-07-19 23:08:13] server_args=ServerArgs(model_path='allenai/olmOCR-7B-0225-preview', tokenizer_path='allenai/olmOCR-7B-0225-preview', tokenizer_mode='auto', load_format='auto', trust_remote_code=False, dtype='auto', kv_cache_dtype='auto', quantization_param_path=None, quantization=None, context_length=None, device='cuda', served_model_name='allenai/olmOCR-7B-0225-preview', chat_template='qwen2-vl', is_embedding=False, revision=None, skip_tokenizer_init=False, host='127.0.0.1', port=30024, mem_fraction_static=0.8, max_running_requests=None, max_total_tokens=None, chunked_prefill_size=2048, max_prefill_tokens=16384, schedule_policy='lpm', schedule_conservativeness=1.0, cpu_offload_gb=0, prefill_only_one_req=False, tp_size=1, stream_interval=1, stream_output=False, random_seed=266199639, constrained_json_whitespace_pattern=None, watchdog_timeout=300, download_dir=None, base_gpu_id=0, log_level='info', log_level_http='warning', log_requests=False, show_time_cost=False, enable_metrics=False, decode_log_interval=40, api_key=None, file_storage_pth='sglang_storage', enable_cache_report=False, dp_size=1, load_balance_method='round_robin', ep_size=1, dist_init_addr=None, nnodes=1, node_rank=0, json_model_override_args='{}', lora_paths=None, max_loras_per_batch=8, attention_backend='flashinfer', sampling_backend='flashinfer', grammar_backend='outlines', speculative_draft_model_path=None, speculative_algorithm=None, speculative_num_steps=5, speculative_num_draft_tokens=64, speculative_eagle_topk=8, enable_double_sparsity=False, ds_channel_config_path=None, ds_heavy_channel_num=32, ds_heavy_token_num=256, ds_heavy_channel_type='qk', ds_sparse_decode_threshold=4096, disable_radix_cache=False, disable_jump_forward=False, disable_cuda_graph=False, disable_cuda_graph_padding=False, disable_outlines_disk_cache=False, disable_custom_all_reduce=False, disable_mla=False, disable_overlap_schedule=False, enable_mixed_chunk=False, enable_dp_attention=False, enable_ep_moe=False, enable_torch_compile=False, torch_compile_max_bs=32, cuda_graph_max_bs=8, cuda_graph_bs=None, torchao_config='', enable_nan_detection=False, enable_p2p_check=False, triton_attention_reduce_in_fp32=False, triton_attention_num_kv_splits=8, num_continuous_decode_steps=1, delete_ckpt_after_loading=False, enable_memory_saver=False, allow_auto_truncate=False, enable_custom_logit_processor=False, tool_call_parser=None) 2025-07-19 23:08:17,243 - __main__ - WARNING - Attempt 2: Please wait for sglang server to become ready... 2025-07-19 23:08:23,178 - sglang - INFO - [2025-07-19 23:08:23] Use chat template for the OpenAI-compatible API server: qwen2-vl 2025-07-19 23:08:23,178 - __main__ - INFO - [2025-07-19 23:08:23] Use chat template for the OpenAI-compatible API server: qwen2-vl 2025-07-19 23:08:23,301 - __main__ - WARNING - Attempt 3: Please wait for sglang server to become ready... 2025-07-19 23:08:29,383 - __main__ - WARNING - Attempt 4: Please wait for sglang server to become ready... 2025-07-19 23:08:30,095 - sglang - INFO - [2025-07-19 23:08:30 TP0] Overlap scheduler is disabled for multimodal models. 2025-07-19 23:08:30,095 - __main__ - INFO - [2025-07-19 23:08:30 TP0] Overlap scheduler is disabled for multimodal models. 2025-07-19 23:08:30,759 - sglang - INFO - [2025-07-19 23:08:30 TP0] Automatically reduce --mem-fraction-static to 0.760 because this is a multimodal model. 2025-07-19 23:08:30,759 - __main__ - INFO - [2025-07-19 23:08:30 TP0] Automatically reduce --mem-fraction-static to 0.760 because this is a multimodal model. 2025-07-19 23:08:30,759 - sglang - INFO - [2025-07-19 23:08:30 TP0] Automatically turn off --chunked-prefill-size and disable radix cache for qwen2-vl. 2025-07-19 23:08:30,759 - __main__ - INFO - [2025-07-19 23:08:30 TP0] Automatically turn off --chunked-prefill-size and disable radix cache for qwen2-vl. 2025-07-19 23:08:30,759 - sglang - INFO - [2025-07-19 23:08:30 TP0] Init torch distributed begin. 2025-07-19 23:08:30,759 - __main__ - INFO - [2025-07-19 23:08:30 TP0] Init torch distributed begin. 2025-07-19 23:08:35,464 - __main__ - WARNING - Attempt 5: Please wait for sglang server to become ready... 2025-07-19 23:08:36,190 - sglang - INFO - [2025-07-19 23:08:36 TP0] Load weight begin. avail mem=23.33 GB 2025-07-19 23:08:36,190 - __main__ - INFO - [2025-07-19 23:08:36 TP0] Load weight begin. avail mem=23.33 GB 2025-07-19 23:08:37,388 - sglang - INFO - [2025-07-19 23:08:37 TP0] Using model weights format ['*.safetensors'] 2025-07-19 23:08:37,388 - __main__ - INFO - [2025-07-19 23:08:37 TP0] Using model weights format ['*.safetensors'] 2025-07-19 23:08:37,969 - sglang - INFO - Loading safetensors checkpoint shards: 0% Completed | 0/4 [00:00 Optional[ScheduleBatch]: 2025-07-19 23:54:50,339 - __main__ - INFO - def get_new_batch_prefill(self) -> Optional[ScheduleBatch]: 2025-07-19 23:54:50,339 - sglang - INFO - 2025-07-19 23:54:50,339 - __main__ - INFO - 2025-07-19 23:54:50,339 - sglang - INFO - KeyboardInterrupt 2025-07-19 23:54:50,339 - __main__ - INFO - KeyboardInterrupt 2025-07-19 23:54:50,340 - __main__ - INFO - Got cancellation request for SGLang server 2025-07-19 23:57:11,442 - __main__ - INFO - Got --pdfs argument, going to add to the work queue 2025-07-19 23:57:11,442 - __main__ - INFO - Loading file at scripts/data/11440000MB2D0234372440125017014.pdf as PDF document 2025-07-19 23:57:11,442 - __main__ - INFO - Found 1 total pdf paths to add 2025-07-19 23:57:11,447 - __main__ - INFO - Calculated items_per_group: 27 based on average pages per PDF: 18.00 2025-07-19 23:57:11,683 - __main__ - INFO - Starting pipeline with PID 563187 2025-07-19 23:57:11,683 - __main__ - INFO - Downloading model with hugging face 'allenai/olmOCR-7B-0225-preview' 2025-07-19 23:57:17,476 - __main__ - WARNING - Attempt 1: Please wait for sglang server to become ready... 2025-07-19 23:57:19,571 - sglang - INFO - [2025-07-19 23:57:19] server_args=ServerArgs(model_path='allenai/olmOCR-7B-0225-preview', tokenizer_path='allenai/olmOCR-7B-0225-preview', tokenizer_mode='auto', load_format='auto', trust_remote_code=False, dtype='auto', kv_cache_dtype='auto', quantization_param_path=None, quantization=None, context_length=None, device='cuda', served_model_name='allenai/olmOCR-7B-0225-preview', chat_template='qwen2-vl', is_embedding=False, revision=None, skip_tokenizer_init=False, host='127.0.0.1', port=30024, mem_fraction_static=0.8, max_running_requests=None, max_total_tokens=None, chunked_prefill_size=2048, max_prefill_tokens=16384, schedule_policy='lpm', schedule_conservativeness=1.0, cpu_offload_gb=0, prefill_only_one_req=False, tp_size=1, stream_interval=1, stream_output=False, random_seed=729530513, constrained_json_whitespace_pattern=None, watchdog_timeout=300, download_dir=None, base_gpu_id=0, log_level='info', log_level_http='warning', log_requests=False, show_time_cost=False, enable_metrics=False, decode_log_interval=40, api_key=None, file_storage_pth='sglang_storage', enable_cache_report=False, dp_size=1, load_balance_method='round_robin', ep_size=1, dist_init_addr=None, nnodes=1, node_rank=0, json_model_override_args='{}', lora_paths=None, max_loras_per_batch=8, attention_backend='flashinfer', sampling_backend='flashinfer', grammar_backend='outlines', speculative_draft_model_path=None, speculative_algorithm=None, speculative_num_steps=5, speculative_num_draft_tokens=64, speculative_eagle_topk=8, enable_double_sparsity=False, ds_channel_config_path=None, ds_heavy_channel_num=32, ds_heavy_token_num=256, ds_heavy_channel_type='qk', ds_sparse_decode_threshold=4096, disable_radix_cache=False, disable_jump_forward=False, disable_cuda_graph=False, disable_cuda_graph_padding=False, disable_outlines_disk_cache=False, disable_custom_all_reduce=False, disable_mla=False, disable_overlap_schedule=False, enable_mixed_chunk=False, enable_dp_attention=False, enable_ep_moe=False, enable_torch_compile=False, torch_compile_max_bs=32, cuda_graph_max_bs=8, cuda_graph_bs=None, torchao_config='', enable_nan_detection=False, enable_p2p_check=False, triton_attention_reduce_in_fp32=False, triton_attention_num_kv_splits=8, num_continuous_decode_steps=1, delete_ckpt_after_loading=False, enable_memory_saver=False, allow_auto_truncate=False, enable_custom_logit_processor=False, tool_call_parser=None) 2025-07-19 23:57:19,572 - __main__ - INFO - [2025-07-19 23:57:19] server_args=ServerArgs(model_path='allenai/olmOCR-7B-0225-preview', tokenizer_path='allenai/olmOCR-7B-0225-preview', tokenizer_mode='auto', load_format='auto', trust_remote_code=False, dtype='auto', kv_cache_dtype='auto', quantization_param_path=None, quantization=None, context_length=None, device='cuda', served_model_name='allenai/olmOCR-7B-0225-preview', chat_template='qwen2-vl', is_embedding=False, revision=None, skip_tokenizer_init=False, host='127.0.0.1', port=30024, mem_fraction_static=0.8, max_running_requests=None, max_total_tokens=None, chunked_prefill_size=2048, max_prefill_tokens=16384, schedule_policy='lpm', schedule_conservativeness=1.0, cpu_offload_gb=0, prefill_only_one_req=False, tp_size=1, stream_interval=1, stream_output=False, random_seed=729530513, constrained_json_whitespace_pattern=None, watchdog_timeout=300, download_dir=None, base_gpu_id=0, log_level='info', log_level_http='warning', log_requests=False, show_time_cost=False, enable_metrics=False, decode_log_interval=40, api_key=None, file_storage_pth='sglang_storage', enable_cache_report=False, dp_size=1, load_balance_method='round_robin', ep_size=1, dist_init_addr=None, nnodes=1, node_rank=0, json_model_override_args='{}', lora_paths=None, max_loras_per_batch=8, attention_backend='flashinfer', sampling_backend='flashinfer', grammar_backend='outlines', speculative_draft_model_path=None, speculative_algorithm=None, speculative_num_steps=5, speculative_num_draft_tokens=64, speculative_eagle_topk=8, enable_double_sparsity=False, ds_channel_config_path=None, ds_heavy_channel_num=32, ds_heavy_token_num=256, ds_heavy_channel_type='qk', ds_sparse_decode_threshold=4096, disable_radix_cache=False, disable_jump_forward=False, disable_cuda_graph=False, disable_cuda_graph_padding=False, disable_outlines_disk_cache=False, disable_custom_all_reduce=False, disable_mla=False, disable_overlap_schedule=False, enable_mixed_chunk=False, enable_dp_attention=False, enable_ep_moe=False, enable_torch_compile=False, torch_compile_max_bs=32, cuda_graph_max_bs=8, cuda_graph_bs=None, torchao_config='', enable_nan_detection=False, enable_p2p_check=False, triton_attention_reduce_in_fp32=False, triton_attention_num_kv_splits=8, num_continuous_decode_steps=1, delete_ckpt_after_loading=False, enable_memory_saver=False, allow_auto_truncate=False, enable_custom_logit_processor=False, tool_call_parser=None) 2025-07-19 23:57:23,609 - __main__ - WARNING - Attempt 2: Please wait for sglang server to become ready... 2025-07-19 23:57:28,966 - sglang - INFO - [2025-07-19 23:57:28] Use chat template for the OpenAI-compatible API server: qwen2-vl 2025-07-19 23:57:28,966 - __main__ - INFO - [2025-07-19 23:57:28] Use chat template for the OpenAI-compatible API server: qwen2-vl 2025-07-19 23:57:29,709 - __main__ - WARNING - Attempt 3: Please wait for sglang server to become ready... 2025-07-19 23:57:30,229 - sglang - INFO - [2025-07-19 23:57:30 TP0] Overlap scheduler is disabled for multimodal models. 2025-07-19 23:57:30,229 - __main__ - INFO - [2025-07-19 23:57:30 TP0] Overlap scheduler is disabled for multimodal models. 2025-07-19 23:57:30,904 - sglang - INFO - [2025-07-19 23:57:30 TP0] Automatically reduce --mem-fraction-static to 0.760 because this is a multimodal model. 2025-07-19 23:57:30,904 - __main__ - INFO - [2025-07-19 23:57:30 TP0] Automatically reduce --mem-fraction-static to 0.760 because this is a multimodal model. 2025-07-19 23:57:30,904 - sglang - INFO - [2025-07-19 23:57:30 TP0] Automatically turn off --chunked-prefill-size and disable radix cache for qwen2-vl. 2025-07-19 23:57:30,904 - __main__ - INFO - [2025-07-19 23:57:30 TP0] Automatically turn off --chunked-prefill-size and disable radix cache for qwen2-vl. 2025-07-19 23:57:30,904 - sglang - INFO - [2025-07-19 23:57:30 TP0] Init torch distributed begin. 2025-07-19 23:57:30,905 - __main__ - INFO - [2025-07-19 23:57:30 TP0] Init torch distributed begin. 2025-07-19 23:57:35,789 - __main__ - WARNING - Attempt 4: Please wait for sglang server to become ready... 2025-07-19 23:57:36,344 - sglang - INFO - [2025-07-19 23:57:36 TP0] Load weight begin. avail mem=23.33 GB 2025-07-19 23:57:36,344 - __main__ - INFO - [2025-07-19 23:57:36 TP0] Load weight begin. avail mem=23.33 GB 2025-07-19 23:57:37,496 - sglang - INFO - [2025-07-19 23:57:37 TP0] Using model weights format ['*.safetensors'] 2025-07-19 23:57:37,496 - __main__ - INFO - [2025-07-19 23:57:37 TP0] Using model weights format ['*.safetensors'] 2025-07-19 23:57:39,017 - sglang - INFO - Loading safetensors checkpoint shards: 0% Completed | 0/4 [00:00: Failed to establish a new connection: [Errno 101] Network is unreachable 2025-07-20 11:15:42,234 - __main__ - INFO - urllib3.exceptions.NewConnectionError: : Failed to establish a new connection: [Errno 101] Network is unreachable 2025-07-20 11:15:42,234 - sglang - INFO - 2025-07-20 11:15:42,234 - __main__ - INFO - 2025-07-20 11:15:42,234 - sglang - INFO - The above exception was the direct cause of the following exception: 2025-07-20 11:15:42,234 - __main__ - INFO - The above exception was the direct cause of the following exception: 2025-07-20 11:15:42,234 - sglang - INFO - 2025-07-20 11:15:42,234 - __main__ - INFO - 2025-07-20 11:15:42,234 - sglang - INFO - Traceback (most recent call last): 2025-07-20 11:15:42,235 - __main__ - INFO - Traceback (most recent call last): 2025-07-20 11:15:42,235 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/requests/adapters.py", line 667, in send 2025-07-20 11:15:42,235 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/requests/adapters.py", line 667, in send 2025-07-20 11:15:42,235 - sglang - INFO - resp = conn.urlopen( 2025-07-20 11:15:42,235 - __main__ - INFO - resp = conn.urlopen( 2025-07-20 11:15:42,235 - sglang - INFO - ^^^^^^^^^^^^^ 2025-07-20 11:15:42,235 - __main__ - INFO - ^^^^^^^^^^^^^ 2025-07-20 11:15:42,235 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/urllib3/connectionpool.py", line 841, in urlopen 2025-07-20 11:15:42,235 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/urllib3/connectionpool.py", line 841, in urlopen 2025-07-20 11:15:42,235 - sglang - INFO - retries = retries.increment( 2025-07-20 11:15:42,235 - __main__ - INFO - retries = retries.increment( 2025-07-20 11:15:42,235 - sglang - INFO - ^^^^^^^^^^^^^^^^^^ 2025-07-20 11:15:42,235 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^ 2025-07-20 11:15:42,235 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/urllib3/util/retry.py", line 519, in increment 2025-07-20 11:15:42,236 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/urllib3/util/retry.py", line 519, in increment 2025-07-20 11:15:42,236 - sglang - INFO - raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] 2025-07-20 11:15:42,236 - __main__ - INFO - raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] 2025-07-20 11:15:42,236 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:15:42,236 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:15:42,236 - sglang - INFO - urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='huggingface.co', port=443): Max retries exceeded with url: /api/models/allenai/olmOCR-7B-0225-preview (Caused by NewConnectionError(': Failed to establish a new connection: [Errno 101] Network is unreachable')) 2025-07-20 11:15:42,236 - __main__ - INFO - urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='huggingface.co', port=443): Max retries exceeded with url: /api/models/allenai/olmOCR-7B-0225-preview (Caused by NewConnectionError(': Failed to establish a new connection: [Errno 101] Network is unreachable')) 2025-07-20 11:15:42,236 - sglang - INFO - 2025-07-20 11:15:42,236 - __main__ - INFO - 2025-07-20 11:15:42,236 - sglang - INFO - During handling of the above exception, another exception occurred: 2025-07-20 11:15:42,236 - __main__ - INFO - During handling of the above exception, another exception occurred: 2025-07-20 11:15:42,236 - sglang - INFO - 2025-07-20 11:15:42,236 - __main__ - INFO - 2025-07-20 11:15:42,237 - sglang - INFO - Traceback (most recent call last): 2025-07-20 11:15:42,237 - __main__ - INFO - Traceback (most recent call last): 2025-07-20 11:15:42,237 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/managers/scheduler.py", line 1773, in run_scheduler_process 2025-07-20 11:15:42,237 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/managers/scheduler.py", line 1773, in run_scheduler_process 2025-07-20 11:15:42,237 - sglang - INFO - scheduler = Scheduler(server_args, port_args, gpu_id, tp_rank, dp_rank) 2025-07-20 11:15:42,237 - __main__ - INFO - scheduler = Scheduler(server_args, port_args, gpu_id, tp_rank, dp_rank) 2025-07-20 11:15:42,237 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:15:42,237 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:15:42,237 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/managers/scheduler.py", line 239, in __init__ 2025-07-20 11:15:42,237 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/managers/scheduler.py", line 239, in __init__ 2025-07-20 11:15:42,237 - sglang - INFO - self.tp_worker = TpWorkerClass( 2025-07-20 11:15:42,237 - __main__ - INFO - self.tp_worker = TpWorkerClass( 2025-07-20 11:15:42,237 - sglang - INFO - ^^^^^^^^^^^^^^ 2025-07-20 11:15:42,238 - __main__ - INFO - ^^^^^^^^^^^^^^ 2025-07-20 11:15:42,238 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/managers/tp_worker.py", line 68, in __init__ 2025-07-20 11:15:42,238 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/managers/tp_worker.py", line 68, in __init__ 2025-07-20 11:15:42,238 - sglang - INFO - self.model_runner = ModelRunner( 2025-07-20 11:15:42,238 - __main__ - INFO - self.model_runner = ModelRunner( 2025-07-20 11:15:42,238 - sglang - INFO - ^^^^^^^^^^^^ 2025-07-20 11:15:42,238 - __main__ - INFO - ^^^^^^^^^^^^ 2025-07-20 11:15:42,238 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/model_executor/model_runner.py", line 185, in __init__ 2025-07-20 11:15:42,238 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/model_executor/model_runner.py", line 185, in __init__ 2025-07-20 11:15:42,238 - sglang - INFO - self.load_model() 2025-07-20 11:15:42,238 - __main__ - INFO - self.load_model() 2025-07-20 11:15:42,238 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/model_executor/model_runner.py", line 306, in load_model 2025-07-20 11:15:42,238 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/model_executor/model_runner.py", line 306, in load_model 2025-07-20 11:15:42,239 - sglang - INFO - self.model = get_model( 2025-07-20 11:15:42,239 - __main__ - INFO - self.model = get_model( 2025-07-20 11:15:42,239 - sglang - INFO - ^^^^^^^^^^ 2025-07-20 11:15:42,239 - __main__ - INFO - ^^^^^^^^^^ 2025-07-20 11:15:42,239 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/model_loader/__init__.py", line 22, in get_model 2025-07-20 11:15:42,239 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/model_loader/__init__.py", line 22, in get_model 2025-07-20 11:15:42,239 - sglang - INFO - return loader.load_model( 2025-07-20 11:15:42,239 - __main__ - INFO - return loader.load_model( 2025-07-20 11:15:42,239 - sglang - INFO - ^^^^^^^^^^^^^^^^^^ 2025-07-20 11:15:42,239 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^ 2025-07-20 11:15:42,239 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/model_loader/loader.py", line 362, in load_model 2025-07-20 11:15:42,239 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/model_loader/loader.py", line 362, in load_model 2025-07-20 11:15:42,239 - sglang - INFO - model.load_weights(self._get_all_weights(model_config, model)) 2025-07-20 11:15:42,239 - __main__ - INFO - model.load_weights(self._get_all_weights(model_config, model)) 2025-07-20 11:15:42,240 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/models/qwen2_vl.py", line 576, in load_weights 2025-07-20 11:15:42,240 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/models/qwen2_vl.py", line 576, in load_weights 2025-07-20 11:15:42,240 - sglang - INFO - for name, loaded_weight in weights: 2025-07-20 11:15:42,240 - __main__ - INFO - for name, loaded_weight in weights: 2025-07-20 11:15:42,240 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/model_loader/loader.py", line 335, in _get_all_weights 2025-07-20 11:15:42,240 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/model_loader/loader.py", line 335, in _get_all_weights 2025-07-20 11:15:42,240 - sglang - INFO - yield from self._get_weights_iterator(primary_weights) 2025-07-20 11:15:42,240 - __main__ - INFO - yield from self._get_weights_iterator(primary_weights) 2025-07-20 11:15:42,240 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:15:42,240 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:15:42,240 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/model_loader/loader.py", line 303, in _get_weights_iterator 2025-07-20 11:15:42,240 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/model_loader/loader.py", line 303, in _get_weights_iterator 2025-07-20 11:15:42,241 - sglang - INFO - hf_folder, hf_weights_files, use_safetensors = self._prepare_weights( 2025-07-20 11:15:42,241 - __main__ - INFO - hf_folder, hf_weights_files, use_safetensors = self._prepare_weights( 2025-07-20 11:15:42,241 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:15:42,241 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:15:42,241 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/model_loader/loader.py", line 255, in _prepare_weights 2025-07-20 11:15:42,241 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/model_loader/loader.py", line 255, in _prepare_weights 2025-07-20 11:15:42,241 - sglang - INFO - hf_folder = download_weights_from_hf( 2025-07-20 11:15:42,241 - __main__ - INFO - hf_folder = download_weights_from_hf( 2025-07-20 11:15:42,241 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:15:42,241 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:15:42,241 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/model_loader/weight_utils.py", line 246, in download_weights_from_hf 2025-07-20 11:15:42,241 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/model_loader/weight_utils.py", line 246, in download_weights_from_hf 2025-07-20 11:15:42,241 - sglang - INFO - file_list = fs.ls(model_name_or_path, detail=False, revision=revision) 2025-07-20 11:15:42,241 - __main__ - INFO - file_list = fs.ls(model_name_or_path, detail=False, revision=revision) 2025-07-20 11:15:42,241 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:15:42,241 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:15:42,241 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/huggingface_hub/hf_file_system.py", line 368, in ls 2025-07-20 11:15:42,241 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/huggingface_hub/hf_file_system.py", line 368, in ls 2025-07-20 11:15:42,241 - sglang - INFO - resolved_path = self.resolve_path(path, revision=revision) 2025-07-20 11:15:42,241 - __main__ - INFO - resolved_path = self.resolve_path(path, revision=revision) 2025-07-20 11:15:42,241 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:15:42,241 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:15:42,241 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/huggingface_hub/hf_file_system.py", line 209, in resolve_path 2025-07-20 11:15:42,241 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/huggingface_hub/hf_file_system.py", line 209, in resolve_path 2025-07-20 11:15:42,241 - sglang - INFO - repo_and_revision_exist, err = self._repo_and_revision_exist(repo_type, repo_id, revision) 2025-07-20 11:15:42,241 - __main__ - INFO - repo_and_revision_exist, err = self._repo_and_revision_exist(repo_type, repo_id, revision) 2025-07-20 11:15:42,241 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:15:42,241 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:15:42,241 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/huggingface_hub/hf_file_system.py", line 125, in _repo_and_revision_exist 2025-07-20 11:15:42,241 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/huggingface_hub/hf_file_system.py", line 125, in _repo_and_revision_exist 2025-07-20 11:15:42,241 - sglang - INFO - self._api.repo_info( 2025-07-20 11:15:42,241 - __main__ - INFO - self._api.repo_info( 2025-07-20 11:15:42,241 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/huggingface_hub/utils/_validators.py", line 114, in _inner_fn 2025-07-20 11:15:42,241 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/huggingface_hub/utils/_validators.py", line 114, in _inner_fn 2025-07-20 11:15:42,242 - sglang - INFO - return fn(*args, **kwargs) 2025-07-20 11:15:42,242 - __main__ - INFO - return fn(*args, **kwargs) 2025-07-20 11:15:42,242 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:15:42,242 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:15:42,242 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/huggingface_hub/hf_api.py", line 2816, in repo_info 2025-07-20 11:15:42,242 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/huggingface_hub/hf_api.py", line 2816, in repo_info 2025-07-20 11:15:42,242 - sglang - INFO - return method( 2025-07-20 11:15:42,242 - __main__ - INFO - return method( 2025-07-20 11:15:42,242 - sglang - INFO - ^^^^^^^ 2025-07-20 11:15:42,242 - __main__ - INFO - ^^^^^^^ 2025-07-20 11:15:42,242 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/huggingface_hub/utils/_validators.py", line 114, in _inner_fn 2025-07-20 11:15:42,242 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/huggingface_hub/utils/_validators.py", line 114, in _inner_fn 2025-07-20 11:15:42,242 - sglang - INFO - return fn(*args, **kwargs) 2025-07-20 11:15:42,242 - __main__ - INFO - return fn(*args, **kwargs) 2025-07-20 11:15:42,242 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:15:42,242 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:15:42,242 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/huggingface_hub/hf_api.py", line 2600, in model_info 2025-07-20 11:15:42,242 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/huggingface_hub/hf_api.py", line 2600, in model_info 2025-07-20 11:15:42,242 - sglang - INFO - r = get_session().get(path, headers=headers, timeout=timeout, params=params) 2025-07-20 11:15:42,242 - __main__ - INFO - r = get_session().get(path, headers=headers, timeout=timeout, params=params) 2025-07-20 11:15:42,242 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:15:42,242 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:15:42,242 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/requests/sessions.py", line 602, in get 2025-07-20 11:15:42,242 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/requests/sessions.py", line 602, in get 2025-07-20 11:15:42,242 - sglang - INFO - return self.request("GET", url, **kwargs) 2025-07-20 11:15:42,242 - __main__ - INFO - return self.request("GET", url, **kwargs) 2025-07-20 11:15:42,242 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:15:42,242 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:15:42,242 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/requests/sessions.py", line 589, in request 2025-07-20 11:15:42,243 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/requests/sessions.py", line 589, in request 2025-07-20 11:15:42,243 - sglang - INFO - resp = self.send(prep, **send_kwargs) 2025-07-20 11:15:42,243 - __main__ - INFO - resp = self.send(prep, **send_kwargs) 2025-07-20 11:15:42,243 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:15:42,243 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:15:42,243 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/requests/sessions.py", line 703, in send 2025-07-20 11:15:42,243 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/requests/sessions.py", line 703, in send 2025-07-20 11:15:42,243 - sglang - INFO - r = adapter.send(request, **kwargs) 2025-07-20 11:15:42,243 - __main__ - INFO - r = adapter.send(request, **kwargs) 2025-07-20 11:15:42,243 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:15:42,243 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:15:42,243 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/huggingface_hub/utils/_http.py", line 96, in send 2025-07-20 11:15:42,243 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/huggingface_hub/utils/_http.py", line 96, in send 2025-07-20 11:15:42,243 - sglang - INFO - return super().send(request, *args, **kwargs) 2025-07-20 11:15:42,243 - __main__ - INFO - return super().send(request, *args, **kwargs) 2025-07-20 11:15:42,243 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:15:42,243 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:15:42,243 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/requests/adapters.py", line 700, in send 2025-07-20 11:15:42,243 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/requests/adapters.py", line 700, in send 2025-07-20 11:15:42,243 - sglang - INFO - raise ConnectionError(e, request=request) 2025-07-20 11:15:42,243 - __main__ - INFO - raise ConnectionError(e, request=request) 2025-07-20 11:15:42,243 - sglang - INFO - requests.exceptions.ConnectionError: (MaxRetryError("HTTPSConnectionPool(host='huggingface.co', port=443): Max retries exceeded with url: /api/models/allenai/olmOCR-7B-0225-preview (Caused by NewConnectionError(': Failed to establish a new connection: [Errno 101] Network is unreachable'))"), '(Request ID: d7db9d3b-7988-48c6-ac50-b06366e3a9c9)') 2025-07-20 11:15:42,243 - __main__ - INFO - requests.exceptions.ConnectionError: (MaxRetryError("HTTPSConnectionPool(host='huggingface.co', port=443): Max retries exceeded with url: /api/models/allenai/olmOCR-7B-0225-preview (Caused by NewConnectionError(': Failed to establish a new connection: [Errno 101] Network is unreachable'))"), '(Request ID: d7db9d3b-7988-48c6-ac50-b06366e3a9c9)') 2025-07-20 11:15:42,243 - sglang - INFO - 2025-07-20 11:15:42,243 - __main__ - INFO - 2025-07-20 11:15:42,244 - sglang - INFO - [2025-07-20 11:15:42] Received sigquit from a child proces. It usually means the child failed. 2025-07-20 11:15:42,244 - __main__ - INFO - [2025-07-20 11:15:42] Received sigquit from a child proces. It usually means the child failed. 2025-07-20 11:15:42,475 - __main__ - WARNING - SGLang server task ended 2025-07-20 11:15:42,528 - __main__ - WARNING - Attempt 24: Please wait for sglang server to become ready... 2025-07-20 11:15:48,512 - sglang - INFO - [2025-07-20 11:15:48] server_args=ServerArgs(model_path='allenai/olmOCR-7B-0225-preview', tokenizer_path='allenai/olmOCR-7B-0225-preview', tokenizer_mode='auto', load_format='auto', trust_remote_code=False, dtype='auto', kv_cache_dtype='auto', quantization_param_path=None, quantization=None, context_length=None, device='cuda', served_model_name='allenai/olmOCR-7B-0225-preview', chat_template='qwen2-vl', is_embedding=False, revision=None, skip_tokenizer_init=False, host='127.0.0.1', port=30024, mem_fraction_static=0.8, max_running_requests=None, max_total_tokens=None, chunked_prefill_size=2048, max_prefill_tokens=16384, schedule_policy='lpm', schedule_conservativeness=1.0, cpu_offload_gb=0, prefill_only_one_req=False, tp_size=1, stream_interval=1, stream_output=False, random_seed=442733111, constrained_json_whitespace_pattern=None, watchdog_timeout=300, download_dir=None, base_gpu_id=0, log_level='info', log_level_http='warning', log_requests=False, show_time_cost=False, enable_metrics=False, decode_log_interval=40, api_key=None, file_storage_pth='sglang_storage', enable_cache_report=False, dp_size=1, load_balance_method='round_robin', ep_size=1, dist_init_addr=None, nnodes=1, node_rank=0, json_model_override_args='{}', lora_paths=None, max_loras_per_batch=8, attention_backend='flashinfer', sampling_backend='flashinfer', grammar_backend='outlines', speculative_draft_model_path=None, speculative_algorithm=None, speculative_num_steps=5, speculative_num_draft_tokens=64, speculative_eagle_topk=8, enable_double_sparsity=False, ds_channel_config_path=None, ds_heavy_channel_num=32, ds_heavy_token_num=256, ds_heavy_channel_type='qk', ds_sparse_decode_threshold=4096, disable_radix_cache=False, disable_jump_forward=False, disable_cuda_graph=False, disable_cuda_graph_padding=False, disable_outlines_disk_cache=False, disable_custom_all_reduce=False, disable_mla=False, disable_overlap_schedule=False, enable_mixed_chunk=False, enable_dp_attention=False, enable_ep_moe=False, enable_torch_compile=False, torch_compile_max_bs=32, cuda_graph_max_bs=8, cuda_graph_bs=None, torchao_config='', enable_nan_detection=False, enable_p2p_check=False, triton_attention_reduce_in_fp32=False, triton_attention_num_kv_splits=8, num_continuous_decode_steps=1, delete_ckpt_after_loading=False, enable_memory_saver=False, allow_auto_truncate=False, enable_custom_logit_processor=False, tool_call_parser=None) 2025-07-20 11:15:48,512 - __main__ - INFO - [2025-07-20 11:15:48] server_args=ServerArgs(model_path='allenai/olmOCR-7B-0225-preview', tokenizer_path='allenai/olmOCR-7B-0225-preview', tokenizer_mode='auto', load_format='auto', trust_remote_code=False, dtype='auto', kv_cache_dtype='auto', quantization_param_path=None, quantization=None, context_length=None, device='cuda', served_model_name='allenai/olmOCR-7B-0225-preview', chat_template='qwen2-vl', is_embedding=False, revision=None, skip_tokenizer_init=False, host='127.0.0.1', port=30024, mem_fraction_static=0.8, max_running_requests=None, max_total_tokens=None, chunked_prefill_size=2048, max_prefill_tokens=16384, schedule_policy='lpm', schedule_conservativeness=1.0, cpu_offload_gb=0, prefill_only_one_req=False, tp_size=1, stream_interval=1, stream_output=False, random_seed=442733111, constrained_json_whitespace_pattern=None, watchdog_timeout=300, download_dir=None, base_gpu_id=0, log_level='info', log_level_http='warning', log_requests=False, show_time_cost=False, enable_metrics=False, decode_log_interval=40, api_key=None, file_storage_pth='sglang_storage', enable_cache_report=False, dp_size=1, load_balance_method='round_robin', ep_size=1, dist_init_addr=None, nnodes=1, node_rank=0, json_model_override_args='{}', lora_paths=None, max_loras_per_batch=8, attention_backend='flashinfer', sampling_backend='flashinfer', grammar_backend='outlines', speculative_draft_model_path=None, speculative_algorithm=None, speculative_num_steps=5, speculative_num_draft_tokens=64, speculative_eagle_topk=8, enable_double_sparsity=False, ds_channel_config_path=None, ds_heavy_channel_num=32, ds_heavy_token_num=256, ds_heavy_channel_type='qk', ds_sparse_decode_threshold=4096, disable_radix_cache=False, disable_jump_forward=False, disable_cuda_graph=False, disable_cuda_graph_padding=False, disable_outlines_disk_cache=False, disable_custom_all_reduce=False, disable_mla=False, disable_overlap_schedule=False, enable_mixed_chunk=False, enable_dp_attention=False, enable_ep_moe=False, enable_torch_compile=False, torch_compile_max_bs=32, cuda_graph_max_bs=8, cuda_graph_bs=None, torchao_config='', enable_nan_detection=False, enable_p2p_check=False, triton_attention_reduce_in_fp32=False, triton_attention_num_kv_splits=8, num_continuous_decode_steps=1, delete_ckpt_after_loading=False, enable_memory_saver=False, allow_auto_truncate=False, enable_custom_logit_processor=False, tool_call_parser=None) 2025-07-20 11:15:48,633 - __main__ - WARNING - Attempt 25: Please wait for sglang server to become ready... 2025-07-20 11:15:54,713 - __main__ - WARNING - Attempt 26: Please wait for sglang server to become ready... 2025-07-20 11:16:00,794 - __main__ - WARNING - Attempt 27: Please wait for sglang server to become ready... 2025-07-20 11:16:06,873 - __main__ - WARNING - Attempt 28: Please wait for sglang server to become ready... 2025-07-20 11:16:12,956 - __main__ - WARNING - Attempt 29: Please wait for sglang server to become ready... 2025-07-20 11:16:19,037 - __main__ - WARNING - Attempt 30: Please wait for sglang server to become ready... 2025-07-20 11:16:25,120 - __main__ - WARNING - Attempt 31: Please wait for sglang server to become ready... 2025-07-20 11:16:31,201 - __main__ - WARNING - Attempt 32: Please wait for sglang server to become ready... 2025-07-20 11:16:37,283 - __main__ - WARNING - Attempt 33: Please wait for sglang server to become ready... 2025-07-20 11:16:43,365 - __main__ - WARNING - Attempt 34: Please wait for sglang server to become ready... 2025-07-20 11:16:49,446 - __main__ - WARNING - Attempt 35: Please wait for sglang server to become ready... 2025-07-20 11:16:55,529 - __main__ - WARNING - Attempt 36: Please wait for sglang server to become ready... 2025-07-20 11:17:01,610 - __main__ - WARNING - Attempt 37: Please wait for sglang server to become ready... 2025-07-20 11:17:07,695 - __main__ - WARNING - Attempt 38: Please wait for sglang server to become ready... 2025-07-20 11:17:13,776 - __main__ - WARNING - Attempt 39: Please wait for sglang server to become ready... 2025-07-20 11:17:19,857 - __main__ - WARNING - Attempt 40: Please wait for sglang server to become ready... 2025-07-20 11:17:25,201 - sglang - INFO - [2025-07-20 11:17:25] Use chat template for the OpenAI-compatible API server: qwen2-vl 2025-07-20 11:17:25,201 - __main__ - INFO - [2025-07-20 11:17:25] Use chat template for the OpenAI-compatible API server: qwen2-vl 2025-07-20 11:17:25,937 - __main__ - WARNING - Attempt 41: Please wait for sglang server to become ready... 2025-07-20 11:17:30,921 - sglang - INFO - [2025-07-20 11:17:30 TP0] Overlap scheduler is disabled for multimodal models. 2025-07-20 11:17:30,921 - __main__ - INFO - [2025-07-20 11:17:30 TP0] Overlap scheduler is disabled for multimodal models. 2025-07-20 11:17:32,018 - __main__ - WARNING - Attempt 42: Please wait for sglang server to become ready... 2025-07-20 11:17:38,101 - __main__ - WARNING - Attempt 43: Please wait for sglang server to become ready... 2025-07-20 11:17:44,181 - __main__ - WARNING - Attempt 44: Please wait for sglang server to become ready... 2025-07-20 11:17:50,263 - __main__ - WARNING - Attempt 45: Please wait for sglang server to become ready... 2025-07-20 11:17:51,072 - sglang - INFO - [2025-07-20 11:17:51 TP0] Automatically reduce --mem-fraction-static to 0.760 because this is a multimodal model. 2025-07-20 11:17:51,072 - __main__ - INFO - [2025-07-20 11:17:51 TP0] Automatically reduce --mem-fraction-static to 0.760 because this is a multimodal model. 2025-07-20 11:17:51,073 - sglang - INFO - [2025-07-20 11:17:51 TP0] Automatically turn off --chunked-prefill-size and disable radix cache for qwen2-vl. 2025-07-20 11:17:51,073 - __main__ - INFO - [2025-07-20 11:17:51 TP0] Automatically turn off --chunked-prefill-size and disable radix cache for qwen2-vl. 2025-07-20 11:17:51,073 - sglang - INFO - [2025-07-20 11:17:51 TP0] Init torch distributed begin. 2025-07-20 11:17:51,073 - __main__ - INFO - [2025-07-20 11:17:51 TP0] Init torch distributed begin. 2025-07-20 11:17:56,343 - __main__ - WARNING - Attempt 46: Please wait for sglang server to become ready... 2025-07-20 11:17:56,482 - sglang - INFO - [2025-07-20 11:17:56 TP0] Load weight begin. avail mem=23.33 GB 2025-07-20 11:17:56,482 - __main__ - INFO - [2025-07-20 11:17:56 TP0] Load weight begin. avail mem=23.33 GB 2025-07-20 11:18:02,424 - __main__ - WARNING - Attempt 47: Please wait for sglang server to become ready... 2025-07-20 11:18:07,166 - sglang - INFO - [2025-07-20 11:18:07 TP0] Scheduler hit an exception: Traceback (most recent call last): 2025-07-20 11:18:07,166 - __main__ - INFO - [2025-07-20 11:18:07 TP0] Scheduler hit an exception: Traceback (most recent call last): 2025-07-20 11:18:07,166 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/urllib3/connection.py", line 198, in _new_conn 2025-07-20 11:18:07,166 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/urllib3/connection.py", line 198, in _new_conn 2025-07-20 11:18:07,166 - sglang - INFO - sock = connection.create_connection( 2025-07-20 11:18:07,166 - __main__ - INFO - sock = connection.create_connection( 2025-07-20 11:18:07,167 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:18:07,167 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:18:07,167 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/urllib3/util/connection.py", line 85, in create_connection 2025-07-20 11:18:07,167 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/urllib3/util/connection.py", line 85, in create_connection 2025-07-20 11:18:07,167 - sglang - INFO - raise err 2025-07-20 11:18:07,167 - __main__ - INFO - raise err 2025-07-20 11:18:07,167 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/urllib3/util/connection.py", line 73, in create_connection 2025-07-20 11:18:07,167 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/urllib3/util/connection.py", line 73, in create_connection 2025-07-20 11:18:07,167 - sglang - INFO - sock.connect(sa) 2025-07-20 11:18:07,167 - __main__ - INFO - sock.connect(sa) 2025-07-20 11:18:07,167 - sglang - INFO - OSError: [Errno 101] Network is unreachable 2025-07-20 11:18:07,168 - __main__ - INFO - OSError: [Errno 101] Network is unreachable 2025-07-20 11:18:07,168 - sglang - INFO - 2025-07-20 11:18:07,168 - __main__ - INFO - 2025-07-20 11:18:07,168 - sglang - INFO - The above exception was the direct cause of the following exception: 2025-07-20 11:18:07,168 - __main__ - INFO - The above exception was the direct cause of the following exception: 2025-07-20 11:18:07,168 - sglang - INFO - 2025-07-20 11:18:07,168 - __main__ - INFO - 2025-07-20 11:18:07,168 - sglang - INFO - Traceback (most recent call last): 2025-07-20 11:18:07,168 - __main__ - INFO - Traceback (most recent call last): 2025-07-20 11:18:07,168 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/urllib3/connectionpool.py", line 787, in urlopen 2025-07-20 11:18:07,168 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/urllib3/connectionpool.py", line 787, in urlopen 2025-07-20 11:18:07,168 - sglang - INFO - response = self._make_request( 2025-07-20 11:18:07,168 - __main__ - INFO - response = self._make_request( 2025-07-20 11:18:07,169 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:18:07,169 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:18:07,169 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/urllib3/connectionpool.py", line 488, in _make_request 2025-07-20 11:18:07,169 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/urllib3/connectionpool.py", line 488, in _make_request 2025-07-20 11:18:07,169 - sglang - INFO - raise new_e 2025-07-20 11:18:07,169 - __main__ - INFO - raise new_e 2025-07-20 11:18:07,169 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/urllib3/connectionpool.py", line 464, in _make_request 2025-07-20 11:18:07,169 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/urllib3/connectionpool.py", line 464, in _make_request 2025-07-20 11:18:07,169 - sglang - INFO - self._validate_conn(conn) 2025-07-20 11:18:07,169 - __main__ - INFO - self._validate_conn(conn) 2025-07-20 11:18:07,169 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/urllib3/connectionpool.py", line 1093, in _validate_conn 2025-07-20 11:18:07,169 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/urllib3/connectionpool.py", line 1093, in _validate_conn 2025-07-20 11:18:07,169 - sglang - INFO - conn.connect() 2025-07-20 11:18:07,170 - __main__ - INFO - conn.connect() 2025-07-20 11:18:07,170 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/urllib3/connection.py", line 704, in connect 2025-07-20 11:18:07,170 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/urllib3/connection.py", line 704, in connect 2025-07-20 11:18:07,170 - sglang - INFO - self.sock = sock = self._new_conn() 2025-07-20 11:18:07,170 - __main__ - INFO - self.sock = sock = self._new_conn() 2025-07-20 11:18:07,170 - sglang - INFO - ^^^^^^^^^^^^^^^^ 2025-07-20 11:18:07,170 - __main__ - INFO - ^^^^^^^^^^^^^^^^ 2025-07-20 11:18:07,170 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/urllib3/connection.py", line 213, in _new_conn 2025-07-20 11:18:07,170 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/urllib3/connection.py", line 213, in _new_conn 2025-07-20 11:18:07,170 - sglang - INFO - raise NewConnectionError( 2025-07-20 11:18:07,170 - __main__ - INFO - raise NewConnectionError( 2025-07-20 11:18:07,170 - sglang - INFO - urllib3.exceptions.NewConnectionError: : Failed to establish a new connection: [Errno 101] Network is unreachable 2025-07-20 11:18:07,170 - __main__ - INFO - urllib3.exceptions.NewConnectionError: : Failed to establish a new connection: [Errno 101] Network is unreachable 2025-07-20 11:18:07,171 - sglang - INFO - 2025-07-20 11:18:07,171 - __main__ - INFO - 2025-07-20 11:18:07,171 - sglang - INFO - The above exception was the direct cause of the following exception: 2025-07-20 11:18:07,171 - __main__ - INFO - The above exception was the direct cause of the following exception: 2025-07-20 11:18:07,171 - sglang - INFO - 2025-07-20 11:18:07,171 - __main__ - INFO - 2025-07-20 11:18:07,171 - sglang - INFO - Traceback (most recent call last): 2025-07-20 11:18:07,171 - __main__ - INFO - Traceback (most recent call last): 2025-07-20 11:18:07,171 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/requests/adapters.py", line 667, in send 2025-07-20 11:18:07,171 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/requests/adapters.py", line 667, in send 2025-07-20 11:18:07,171 - sglang - INFO - resp = conn.urlopen( 2025-07-20 11:18:07,171 - __main__ - INFO - resp = conn.urlopen( 2025-07-20 11:18:07,171 - sglang - INFO - ^^^^^^^^^^^^^ 2025-07-20 11:18:07,171 - __main__ - INFO - ^^^^^^^^^^^^^ 2025-07-20 11:18:07,172 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/urllib3/connectionpool.py", line 841, in urlopen 2025-07-20 11:18:07,172 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/urllib3/connectionpool.py", line 841, in urlopen 2025-07-20 11:18:07,172 - sglang - INFO - retries = retries.increment( 2025-07-20 11:18:07,172 - __main__ - INFO - retries = retries.increment( 2025-07-20 11:18:07,172 - sglang - INFO - ^^^^^^^^^^^^^^^^^^ 2025-07-20 11:18:07,172 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^ 2025-07-20 11:18:07,172 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/urllib3/util/retry.py", line 519, in increment 2025-07-20 11:18:07,172 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/urllib3/util/retry.py", line 519, in increment 2025-07-20 11:18:07,172 - sglang - INFO - raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] 2025-07-20 11:18:07,172 - __main__ - INFO - raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] 2025-07-20 11:18:07,172 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:18:07,172 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:18:07,173 - sglang - INFO - urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='huggingface.co', port=443): Max retries exceeded with url: /api/models/allenai/olmOCR-7B-0225-preview (Caused by NewConnectionError(': Failed to establish a new connection: [Errno 101] Network is unreachable')) 2025-07-20 11:18:07,173 - __main__ - INFO - urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='huggingface.co', port=443): Max retries exceeded with url: /api/models/allenai/olmOCR-7B-0225-preview (Caused by NewConnectionError(': Failed to establish a new connection: [Errno 101] Network is unreachable')) 2025-07-20 11:18:07,173 - sglang - INFO - 2025-07-20 11:18:07,173 - __main__ - INFO - 2025-07-20 11:18:07,173 - sglang - INFO - During handling of the above exception, another exception occurred: 2025-07-20 11:18:07,173 - __main__ - INFO - During handling of the above exception, another exception occurred: 2025-07-20 11:18:07,173 - sglang - INFO - 2025-07-20 11:18:07,173 - __main__ - INFO - 2025-07-20 11:18:07,173 - sglang - INFO - Traceback (most recent call last): 2025-07-20 11:18:07,173 - __main__ - INFO - Traceback (most recent call last): 2025-07-20 11:18:07,173 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/managers/scheduler.py", line 1773, in run_scheduler_process 2025-07-20 11:18:07,173 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/managers/scheduler.py", line 1773, in run_scheduler_process 2025-07-20 11:18:07,173 - sglang - INFO - scheduler = Scheduler(server_args, port_args, gpu_id, tp_rank, dp_rank) 2025-07-20 11:18:07,174 - __main__ - INFO - scheduler = Scheduler(server_args, port_args, gpu_id, tp_rank, dp_rank) 2025-07-20 11:18:07,174 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:18:07,174 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:18:07,174 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/managers/scheduler.py", line 239, in __init__ 2025-07-20 11:18:07,174 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/managers/scheduler.py", line 239, in __init__ 2025-07-20 11:18:07,174 - sglang - INFO - self.tp_worker = TpWorkerClass( 2025-07-20 11:18:07,174 - __main__ - INFO - self.tp_worker = TpWorkerClass( 2025-07-20 11:18:07,174 - sglang - INFO - ^^^^^^^^^^^^^^ 2025-07-20 11:18:07,174 - __main__ - INFO - ^^^^^^^^^^^^^^ 2025-07-20 11:18:07,174 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/managers/tp_worker.py", line 68, in __init__ 2025-07-20 11:18:07,174 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/managers/tp_worker.py", line 68, in __init__ 2025-07-20 11:18:07,174 - sglang - INFO - self.model_runner = ModelRunner( 2025-07-20 11:18:07,174 - __main__ - INFO - self.model_runner = ModelRunner( 2025-07-20 11:18:07,175 - sglang - INFO - ^^^^^^^^^^^^ 2025-07-20 11:18:07,175 - __main__ - INFO - ^^^^^^^^^^^^ 2025-07-20 11:18:07,175 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/model_executor/model_runner.py", line 185, in __init__ 2025-07-20 11:18:07,175 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/model_executor/model_runner.py", line 185, in __init__ 2025-07-20 11:18:07,175 - sglang - INFO - self.load_model() 2025-07-20 11:18:07,175 - __main__ - INFO - self.load_model() 2025-07-20 11:18:07,175 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/model_executor/model_runner.py", line 306, in load_model 2025-07-20 11:18:07,175 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/model_executor/model_runner.py", line 306, in load_model 2025-07-20 11:18:07,175 - sglang - INFO - self.model = get_model( 2025-07-20 11:18:07,175 - __main__ - INFO - self.model = get_model( 2025-07-20 11:18:07,175 - sglang - INFO - ^^^^^^^^^^ 2025-07-20 11:18:07,175 - __main__ - INFO - ^^^^^^^^^^ 2025-07-20 11:18:07,175 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/model_loader/__init__.py", line 22, in get_model 2025-07-20 11:18:07,175 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/model_loader/__init__.py", line 22, in get_model 2025-07-20 11:18:07,176 - sglang - INFO - return loader.load_model( 2025-07-20 11:18:07,176 - __main__ - INFO - return loader.load_model( 2025-07-20 11:18:07,176 - sglang - INFO - ^^^^^^^^^^^^^^^^^^ 2025-07-20 11:18:07,176 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^ 2025-07-20 11:18:07,176 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/model_loader/loader.py", line 362, in load_model 2025-07-20 11:18:07,176 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/model_loader/loader.py", line 362, in load_model 2025-07-20 11:18:07,176 - sglang - INFO - model.load_weights(self._get_all_weights(model_config, model)) 2025-07-20 11:18:07,176 - __main__ - INFO - model.load_weights(self._get_all_weights(model_config, model)) 2025-07-20 11:18:07,176 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/models/qwen2_vl.py", line 576, in load_weights 2025-07-20 11:18:07,176 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/models/qwen2_vl.py", line 576, in load_weights 2025-07-20 11:18:07,176 - sglang - INFO - for name, loaded_weight in weights: 2025-07-20 11:18:07,176 - __main__ - INFO - for name, loaded_weight in weights: 2025-07-20 11:18:07,177 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/model_loader/loader.py", line 335, in _get_all_weights 2025-07-20 11:18:07,177 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/model_loader/loader.py", line 335, in _get_all_weights 2025-07-20 11:18:07,177 - sglang - INFO - yield from self._get_weights_iterator(primary_weights) 2025-07-20 11:18:07,177 - __main__ - INFO - yield from self._get_weights_iterator(primary_weights) 2025-07-20 11:18:07,177 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:18:07,177 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:18:07,177 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/model_loader/loader.py", line 303, in _get_weights_iterator 2025-07-20 11:18:07,177 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/model_loader/loader.py", line 303, in _get_weights_iterator 2025-07-20 11:18:07,177 - sglang - INFO - hf_folder, hf_weights_files, use_safetensors = self._prepare_weights( 2025-07-20 11:18:07,177 - __main__ - INFO - hf_folder, hf_weights_files, use_safetensors = self._prepare_weights( 2025-07-20 11:18:07,177 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:18:07,177 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:18:07,177 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/model_loader/loader.py", line 255, in _prepare_weights 2025-07-20 11:18:07,178 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/model_loader/loader.py", line 255, in _prepare_weights 2025-07-20 11:18:07,178 - sglang - INFO - hf_folder = download_weights_from_hf( 2025-07-20 11:18:07,178 - __main__ - INFO - hf_folder = download_weights_from_hf( 2025-07-20 11:18:07,178 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:18:07,178 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:18:07,178 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/model_loader/weight_utils.py", line 246, in download_weights_from_hf 2025-07-20 11:18:07,178 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/model_loader/weight_utils.py", line 246, in download_weights_from_hf 2025-07-20 11:18:07,178 - sglang - INFO - file_list = fs.ls(model_name_or_path, detail=False, revision=revision) 2025-07-20 11:18:07,178 - __main__ - INFO - file_list = fs.ls(model_name_or_path, detail=False, revision=revision) 2025-07-20 11:18:07,178 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:18:07,178 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:18:07,180 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/huggingface_hub/hf_file_system.py", line 368, in ls 2025-07-20 11:18:07,180 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/huggingface_hub/hf_file_system.py", line 368, in ls 2025-07-20 11:18:07,180 - sglang - INFO - resolved_path = self.resolve_path(path, revision=revision) 2025-07-20 11:18:07,180 - __main__ - INFO - resolved_path = self.resolve_path(path, revision=revision) 2025-07-20 11:18:07,180 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:18:07,180 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:18:07,180 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/huggingface_hub/hf_file_system.py", line 209, in resolve_path 2025-07-20 11:18:07,180 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/huggingface_hub/hf_file_system.py", line 209, in resolve_path 2025-07-20 11:18:07,180 - sglang - INFO - repo_and_revision_exist, err = self._repo_and_revision_exist(repo_type, repo_id, revision) 2025-07-20 11:18:07,180 - __main__ - INFO - repo_and_revision_exist, err = self._repo_and_revision_exist(repo_type, repo_id, revision) 2025-07-20 11:18:07,180 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:18:07,180 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:18:07,180 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/huggingface_hub/hf_file_system.py", line 125, in _repo_and_revision_exist 2025-07-20 11:18:07,180 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/huggingface_hub/hf_file_system.py", line 125, in _repo_and_revision_exist 2025-07-20 11:18:07,180 - sglang - INFO - self._api.repo_info( 2025-07-20 11:18:07,180 - __main__ - INFO - self._api.repo_info( 2025-07-20 11:18:07,180 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/huggingface_hub/utils/_validators.py", line 114, in _inner_fn 2025-07-20 11:18:07,180 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/huggingface_hub/utils/_validators.py", line 114, in _inner_fn 2025-07-20 11:18:07,180 - sglang - INFO - return fn(*args, **kwargs) 2025-07-20 11:18:07,180 - __main__ - INFO - return fn(*args, **kwargs) 2025-07-20 11:18:07,180 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:18:07,180 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:18:07,180 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/huggingface_hub/hf_api.py", line 2816, in repo_info 2025-07-20 11:18:07,180 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/huggingface_hub/hf_api.py", line 2816, in repo_info 2025-07-20 11:18:07,180 - sglang - INFO - return method( 2025-07-20 11:18:07,180 - __main__ - INFO - return method( 2025-07-20 11:18:07,180 - sglang - INFO - ^^^^^^^ 2025-07-20 11:18:07,180 - __main__ - INFO - ^^^^^^^ 2025-07-20 11:18:07,181 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/huggingface_hub/utils/_validators.py", line 114, in _inner_fn 2025-07-20 11:18:07,181 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/huggingface_hub/utils/_validators.py", line 114, in _inner_fn 2025-07-20 11:18:07,181 - sglang - INFO - return fn(*args, **kwargs) 2025-07-20 11:18:07,233 - __main__ - INFO - return fn(*args, **kwargs) 2025-07-20 11:18:07,233 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:18:07,233 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:18:07,233 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/huggingface_hub/hf_api.py", line 2600, in model_info 2025-07-20 11:18:07,233 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/huggingface_hub/hf_api.py", line 2600, in model_info 2025-07-20 11:18:07,233 - sglang - INFO - r = get_session().get(path, headers=headers, timeout=timeout, params=params) 2025-07-20 11:18:07,233 - __main__ - INFO - r = get_session().get(path, headers=headers, timeout=timeout, params=params) 2025-07-20 11:18:07,233 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:18:07,233 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:18:07,233 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/requests/sessions.py", line 602, in get 2025-07-20 11:18:07,233 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/requests/sessions.py", line 602, in get 2025-07-20 11:18:07,234 - sglang - INFO - return self.request("GET", url, **kwargs) 2025-07-20 11:18:07,234 - __main__ - INFO - return self.request("GET", url, **kwargs) 2025-07-20 11:18:07,234 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:18:07,234 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:18:07,234 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/requests/sessions.py", line 589, in request 2025-07-20 11:18:07,234 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/requests/sessions.py", line 589, in request 2025-07-20 11:18:07,234 - sglang - INFO - resp = self.send(prep, **send_kwargs) 2025-07-20 11:18:07,234 - __main__ - INFO - resp = self.send(prep, **send_kwargs) 2025-07-20 11:18:07,234 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:18:07,234 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:18:07,234 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/requests/sessions.py", line 703, in send 2025-07-20 11:18:07,234 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/requests/sessions.py", line 703, in send 2025-07-20 11:18:07,234 - sglang - INFO - r = adapter.send(request, **kwargs) 2025-07-20 11:18:07,234 - __main__ - INFO - r = adapter.send(request, **kwargs) 2025-07-20 11:18:07,234 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:18:07,234 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:18:07,234 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/huggingface_hub/utils/_http.py", line 96, in send 2025-07-20 11:18:07,234 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/huggingface_hub/utils/_http.py", line 96, in send 2025-07-20 11:18:07,234 - sglang - INFO - return super().send(request, *args, **kwargs) 2025-07-20 11:18:07,234 - __main__ - INFO - return super().send(request, *args, **kwargs) 2025-07-20 11:18:07,234 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:18:07,234 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:18:07,234 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/requests/adapters.py", line 700, in send 2025-07-20 11:18:07,234 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/requests/adapters.py", line 700, in send 2025-07-20 11:18:07,234 - sglang - INFO - raise ConnectionError(e, request=request) 2025-07-20 11:18:07,234 - __main__ - INFO - raise ConnectionError(e, request=request) 2025-07-20 11:18:07,234 - sglang - INFO - requests.exceptions.ConnectionError: (MaxRetryError("HTTPSConnectionPool(host='huggingface.co', port=443): Max retries exceeded with url: /api/models/allenai/olmOCR-7B-0225-preview (Caused by NewConnectionError(': Failed to establish a new connection: [Errno 101] Network is unreachable'))"), '(Request ID: 0e0286ef-566f-4e0f-8c78-0db3717091a5)') 2025-07-20 11:18:07,234 - __main__ - INFO - requests.exceptions.ConnectionError: (MaxRetryError("HTTPSConnectionPool(host='huggingface.co', port=443): Max retries exceeded with url: /api/models/allenai/olmOCR-7B-0225-preview (Caused by NewConnectionError(': Failed to establish a new connection: [Errno 101] Network is unreachable'))"), '(Request ID: 0e0286ef-566f-4e0f-8c78-0db3717091a5)') 2025-07-20 11:18:07,234 - sglang - INFO - 2025-07-20 11:18:07,234 - __main__ - INFO - 2025-07-20 11:18:07,235 - sglang - INFO - [2025-07-20 11:18:07] Received sigquit from a child proces. It usually means the child failed. 2025-07-20 11:18:07,235 - __main__ - INFO - [2025-07-20 11:18:07] Received sigquit from a child proces. It usually means the child failed. 2025-07-20 11:18:07,541 - __main__ - WARNING - SGLang server task ended 2025-07-20 11:18:08,506 - __main__ - WARNING - Attempt 48: Please wait for sglang server to become ready... 2025-07-20 11:18:14,589 - __main__ - WARNING - Attempt 49: Please wait for sglang server to become ready... 2025-07-20 11:18:14,891 - sglang - INFO - [2025-07-20 11:18:14] server_args=ServerArgs(model_path='allenai/olmOCR-7B-0225-preview', tokenizer_path='allenai/olmOCR-7B-0225-preview', tokenizer_mode='auto', load_format='auto', trust_remote_code=False, dtype='auto', kv_cache_dtype='auto', quantization_param_path=None, quantization=None, context_length=None, device='cuda', served_model_name='allenai/olmOCR-7B-0225-preview', chat_template='qwen2-vl', is_embedding=False, revision=None, skip_tokenizer_init=False, host='127.0.0.1', port=30024, mem_fraction_static=0.8, max_running_requests=None, max_total_tokens=None, chunked_prefill_size=2048, max_prefill_tokens=16384, schedule_policy='lpm', schedule_conservativeness=1.0, cpu_offload_gb=0, prefill_only_one_req=False, tp_size=1, stream_interval=1, stream_output=False, random_seed=543652995, constrained_json_whitespace_pattern=None, watchdog_timeout=300, download_dir=None, base_gpu_id=0, log_level='info', log_level_http='warning', log_requests=False, show_time_cost=False, enable_metrics=False, decode_log_interval=40, api_key=None, file_storage_pth='sglang_storage', enable_cache_report=False, dp_size=1, load_balance_method='round_robin', ep_size=1, dist_init_addr=None, nnodes=1, node_rank=0, json_model_override_args='{}', lora_paths=None, max_loras_per_batch=8, attention_backend='flashinfer', sampling_backend='flashinfer', grammar_backend='outlines', speculative_draft_model_path=None, speculative_algorithm=None, speculative_num_steps=5, speculative_num_draft_tokens=64, speculative_eagle_topk=8, enable_double_sparsity=False, ds_channel_config_path=None, ds_heavy_channel_num=32, ds_heavy_token_num=256, ds_heavy_channel_type='qk', ds_sparse_decode_threshold=4096, disable_radix_cache=False, disable_jump_forward=False, disable_cuda_graph=False, disable_cuda_graph_padding=False, disable_outlines_disk_cache=False, disable_custom_all_reduce=False, disable_mla=False, disable_overlap_schedule=False, enable_mixed_chunk=False, enable_dp_attention=False, enable_ep_moe=False, enable_torch_compile=False, torch_compile_max_bs=32, cuda_graph_max_bs=8, cuda_graph_bs=None, torchao_config='', enable_nan_detection=False, enable_p2p_check=False, triton_attention_reduce_in_fp32=False, triton_attention_num_kv_splits=8, num_continuous_decode_steps=1, delete_ckpt_after_loading=False, enable_memory_saver=False, allow_auto_truncate=False, enable_custom_logit_processor=False, tool_call_parser=None) 2025-07-20 11:18:14,891 - __main__ - INFO - [2025-07-20 11:18:14] server_args=ServerArgs(model_path='allenai/olmOCR-7B-0225-preview', tokenizer_path='allenai/olmOCR-7B-0225-preview', tokenizer_mode='auto', load_format='auto', trust_remote_code=False, dtype='auto', kv_cache_dtype='auto', quantization_param_path=None, quantization=None, context_length=None, device='cuda', served_model_name='allenai/olmOCR-7B-0225-preview', chat_template='qwen2-vl', is_embedding=False, revision=None, skip_tokenizer_init=False, host='127.0.0.1', port=30024, mem_fraction_static=0.8, max_running_requests=None, max_total_tokens=None, chunked_prefill_size=2048, max_prefill_tokens=16384, schedule_policy='lpm', schedule_conservativeness=1.0, cpu_offload_gb=0, prefill_only_one_req=False, tp_size=1, stream_interval=1, stream_output=False, random_seed=543652995, constrained_json_whitespace_pattern=None, watchdog_timeout=300, download_dir=None, base_gpu_id=0, log_level='info', log_level_http='warning', log_requests=False, show_time_cost=False, enable_metrics=False, decode_log_interval=40, api_key=None, file_storage_pth='sglang_storage', enable_cache_report=False, dp_size=1, load_balance_method='round_robin', ep_size=1, dist_init_addr=None, nnodes=1, node_rank=0, json_model_override_args='{}', lora_paths=None, max_loras_per_batch=8, attention_backend='flashinfer', sampling_backend='flashinfer', grammar_backend='outlines', speculative_draft_model_path=None, speculative_algorithm=None, speculative_num_steps=5, speculative_num_draft_tokens=64, speculative_eagle_topk=8, enable_double_sparsity=False, ds_channel_config_path=None, ds_heavy_channel_num=32, ds_heavy_token_num=256, ds_heavy_channel_type='qk', ds_sparse_decode_threshold=4096, disable_radix_cache=False, disable_jump_forward=False, disable_cuda_graph=False, disable_cuda_graph_padding=False, disable_outlines_disk_cache=False, disable_custom_all_reduce=False, disable_mla=False, disable_overlap_schedule=False, enable_mixed_chunk=False, enable_dp_attention=False, enable_ep_moe=False, enable_torch_compile=False, torch_compile_max_bs=32, cuda_graph_max_bs=8, cuda_graph_bs=None, torchao_config='', enable_nan_detection=False, enable_p2p_check=False, triton_attention_reduce_in_fp32=False, triton_attention_num_kv_splits=8, num_continuous_decode_steps=1, delete_ckpt_after_loading=False, enable_memory_saver=False, allow_auto_truncate=False, enable_custom_logit_processor=False, tool_call_parser=None) 2025-07-20 11:18:20,733 - __main__ - WARNING - Attempt 50: Please wait for sglang server to become ready... 2025-07-20 11:18:26,815 - __main__ - WARNING - Attempt 51: Please wait for sglang server to become ready... 2025-07-20 11:18:32,896 - __main__ - WARNING - Attempt 52: Please wait for sglang server to become ready... 2025-07-20 11:18:38,976 - __main__ - WARNING - Attempt 53: Please wait for sglang server to become ready... 2025-07-20 11:18:45,057 - __main__ - WARNING - Attempt 54: Please wait for sglang server to become ready... 2025-07-20 11:18:51,137 - __main__ - WARNING - Attempt 55: Please wait for sglang server to become ready... 2025-07-20 11:18:57,220 - __main__ - WARNING - Attempt 56: Please wait for sglang server to become ready... 2025-07-20 11:19:03,309 - __main__ - WARNING - Attempt 57: Please wait for sglang server to become ready... 2025-07-20 11:19:09,392 - __main__ - WARNING - Attempt 58: Please wait for sglang server to become ready... 2025-07-20 11:19:15,472 - __main__ - WARNING - Attempt 59: Please wait for sglang server to become ready... 2025-07-20 11:19:21,553 - __main__ - WARNING - Attempt 60: Please wait for sglang server to become ready... 2025-07-20 11:19:27,636 - __main__ - WARNING - Attempt 61: Please wait for sglang server to become ready... 2025-07-20 11:19:33,716 - __main__ - WARNING - Attempt 62: Please wait for sglang server to become ready... 2025-07-20 11:19:39,797 - __main__ - WARNING - Attempt 63: Please wait for sglang server to become ready... 2025-07-20 11:19:45,873 - __main__ - WARNING - Attempt 64: Please wait for sglang server to become ready... 2025-07-20 11:19:51,665 - sglang - INFO - [2025-07-20 11:19:51] Use chat template for the OpenAI-compatible API server: qwen2-vl 2025-07-20 11:19:51,665 - __main__ - INFO - [2025-07-20 11:19:51] Use chat template for the OpenAI-compatible API server: qwen2-vl 2025-07-20 11:19:51,960 - __main__ - WARNING - Attempt 65: Please wait for sglang server to become ready... 2025-07-20 11:19:58,041 - __main__ - WARNING - Attempt 66: Please wait for sglang server to become ready... 2025-07-20 11:19:58,086 - sglang - INFO - [2025-07-20 11:19:58 TP0] Overlap scheduler is disabled for multimodal models. 2025-07-20 11:19:58,087 - __main__ - INFO - [2025-07-20 11:19:58 TP0] Overlap scheduler is disabled for multimodal models. 2025-07-20 11:20:04,124 - __main__ - WARNING - Attempt 67: Please wait for sglang server to become ready... 2025-07-20 11:20:10,205 - __main__ - WARNING - Attempt 68: Please wait for sglang server to become ready... 2025-07-20 11:20:16,285 - __main__ - WARNING - Attempt 69: Please wait for sglang server to become ready... 2025-07-20 11:20:18,243 - sglang - INFO - [2025-07-20 11:20:18 TP0] Automatically reduce --mem-fraction-static to 0.760 because this is a multimodal model. 2025-07-20 11:20:18,243 - __main__ - INFO - [2025-07-20 11:20:18 TP0] Automatically reduce --mem-fraction-static to 0.760 because this is a multimodal model. 2025-07-20 11:20:18,243 - sglang - INFO - [2025-07-20 11:20:18 TP0] Automatically turn off --chunked-prefill-size and disable radix cache for qwen2-vl. 2025-07-20 11:20:18,243 - __main__ - INFO - [2025-07-20 11:20:18 TP0] Automatically turn off --chunked-prefill-size and disable radix cache for qwen2-vl. 2025-07-20 11:20:18,243 - sglang - INFO - [2025-07-20 11:20:18 TP0] Init torch distributed begin. 2025-07-20 11:20:18,243 - __main__ - INFO - [2025-07-20 11:20:18 TP0] Init torch distributed begin. 2025-07-20 11:20:22,368 - __main__ - WARNING - Attempt 70: Please wait for sglang server to become ready... 2025-07-20 11:20:23,643 - sglang - INFO - [2025-07-20 11:20:23 TP0] Load weight begin. avail mem=23.33 GB 2025-07-20 11:20:23,643 - __main__ - INFO - [2025-07-20 11:20:23 TP0] Load weight begin. avail mem=23.33 GB 2025-07-20 11:20:28,449 - __main__ - WARNING - Attempt 71: Please wait for sglang server to become ready... 2025-07-20 11:20:34,348 - sglang - INFO - [2025-07-20 11:20:34 TP0] Scheduler hit an exception: Traceback (most recent call last): 2025-07-20 11:20:34,349 - __main__ - INFO - [2025-07-20 11:20:34 TP0] Scheduler hit an exception: Traceback (most recent call last): 2025-07-20 11:20:34,349 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/urllib3/connection.py", line 198, in _new_conn 2025-07-20 11:20:34,349 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/urllib3/connection.py", line 198, in _new_conn 2025-07-20 11:20:34,349 - sglang - INFO - sock = connection.create_connection( 2025-07-20 11:20:34,349 - __main__ - INFO - sock = connection.create_connection( 2025-07-20 11:20:34,349 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:20:34,349 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:20:34,349 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/urllib3/util/connection.py", line 85, in create_connection 2025-07-20 11:20:34,349 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/urllib3/util/connection.py", line 85, in create_connection 2025-07-20 11:20:34,349 - sglang - INFO - raise err 2025-07-20 11:20:34,350 - __main__ - INFO - raise err 2025-07-20 11:20:34,350 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/urllib3/util/connection.py", line 73, in create_connection 2025-07-20 11:20:34,350 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/urllib3/util/connection.py", line 73, in create_connection 2025-07-20 11:20:34,350 - sglang - INFO - sock.connect(sa) 2025-07-20 11:20:34,350 - __main__ - INFO - sock.connect(sa) 2025-07-20 11:20:34,350 - sglang - INFO - OSError: [Errno 101] Network is unreachable 2025-07-20 11:20:34,350 - __main__ - INFO - OSError: [Errno 101] Network is unreachable 2025-07-20 11:20:34,350 - sglang - INFO - 2025-07-20 11:20:34,350 - __main__ - INFO - 2025-07-20 11:20:34,350 - sglang - INFO - The above exception was the direct cause of the following exception: 2025-07-20 11:20:34,350 - __main__ - INFO - The above exception was the direct cause of the following exception: 2025-07-20 11:20:34,350 - sglang - INFO - 2025-07-20 11:20:34,350 - __main__ - INFO - 2025-07-20 11:20:34,351 - sglang - INFO - Traceback (most recent call last): 2025-07-20 11:20:34,351 - __main__ - INFO - Traceback (most recent call last): 2025-07-20 11:20:34,351 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/urllib3/connectionpool.py", line 787, in urlopen 2025-07-20 11:20:34,351 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/urllib3/connectionpool.py", line 787, in urlopen 2025-07-20 11:20:34,351 - sglang - INFO - response = self._make_request( 2025-07-20 11:20:34,351 - __main__ - INFO - response = self._make_request( 2025-07-20 11:20:34,351 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:20:34,351 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:20:34,351 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/urllib3/connectionpool.py", line 488, in _make_request 2025-07-20 11:20:34,351 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/urllib3/connectionpool.py", line 488, in _make_request 2025-07-20 11:20:34,351 - sglang - INFO - raise new_e 2025-07-20 11:20:34,351 - __main__ - INFO - raise new_e 2025-07-20 11:20:34,351 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/urllib3/connectionpool.py", line 464, in _make_request 2025-07-20 11:20:34,352 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/urllib3/connectionpool.py", line 464, in _make_request 2025-07-20 11:20:34,352 - sglang - INFO - self._validate_conn(conn) 2025-07-20 11:20:34,352 - __main__ - INFO - self._validate_conn(conn) 2025-07-20 11:20:34,352 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/urllib3/connectionpool.py", line 1093, in _validate_conn 2025-07-20 11:20:34,352 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/urllib3/connectionpool.py", line 1093, in _validate_conn 2025-07-20 11:20:34,352 - sglang - INFO - conn.connect() 2025-07-20 11:20:34,352 - __main__ - INFO - conn.connect() 2025-07-20 11:20:34,352 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/urllib3/connection.py", line 704, in connect 2025-07-20 11:20:34,352 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/urllib3/connection.py", line 704, in connect 2025-07-20 11:20:34,352 - sglang - INFO - self.sock = sock = self._new_conn() 2025-07-20 11:20:34,352 - __main__ - INFO - self.sock = sock = self._new_conn() 2025-07-20 11:20:34,352 - sglang - INFO - ^^^^^^^^^^^^^^^^ 2025-07-20 11:20:34,352 - __main__ - INFO - ^^^^^^^^^^^^^^^^ 2025-07-20 11:20:34,353 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/urllib3/connection.py", line 213, in _new_conn 2025-07-20 11:20:34,353 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/urllib3/connection.py", line 213, in _new_conn 2025-07-20 11:20:34,353 - sglang - INFO - raise NewConnectionError( 2025-07-20 11:20:34,353 - __main__ - INFO - raise NewConnectionError( 2025-07-20 11:20:34,353 - sglang - INFO - urllib3.exceptions.NewConnectionError: : Failed to establish a new connection: [Errno 101] Network is unreachable 2025-07-20 11:20:34,353 - __main__ - INFO - urllib3.exceptions.NewConnectionError: : Failed to establish a new connection: [Errno 101] Network is unreachable 2025-07-20 11:20:34,353 - sglang - INFO - 2025-07-20 11:20:34,353 - __main__ - INFO - 2025-07-20 11:20:34,353 - sglang - INFO - The above exception was the direct cause of the following exception: 2025-07-20 11:20:34,353 - __main__ - INFO - The above exception was the direct cause of the following exception: 2025-07-20 11:20:34,353 - sglang - INFO - 2025-07-20 11:20:34,353 - __main__ - INFO - 2025-07-20 11:20:34,353 - sglang - INFO - Traceback (most recent call last): 2025-07-20 11:20:34,354 - __main__ - INFO - Traceback (most recent call last): 2025-07-20 11:20:34,354 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/requests/adapters.py", line 667, in send 2025-07-20 11:20:34,354 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/requests/adapters.py", line 667, in send 2025-07-20 11:20:34,354 - sglang - INFO - resp = conn.urlopen( 2025-07-20 11:20:34,354 - __main__ - INFO - resp = conn.urlopen( 2025-07-20 11:20:34,354 - sglang - INFO - ^^^^^^^^^^^^^ 2025-07-20 11:20:34,354 - __main__ - INFO - ^^^^^^^^^^^^^ 2025-07-20 11:20:34,354 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/urllib3/connectionpool.py", line 841, in urlopen 2025-07-20 11:20:34,354 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/urllib3/connectionpool.py", line 841, in urlopen 2025-07-20 11:20:34,354 - sglang - INFO - retries = retries.increment( 2025-07-20 11:20:34,354 - __main__ - INFO - retries = retries.increment( 2025-07-20 11:20:34,354 - sglang - INFO - ^^^^^^^^^^^^^^^^^^ 2025-07-20 11:20:34,354 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^ 2025-07-20 11:20:34,355 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/urllib3/util/retry.py", line 519, in increment 2025-07-20 11:20:34,355 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/urllib3/util/retry.py", line 519, in increment 2025-07-20 11:20:34,355 - sglang - INFO - raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] 2025-07-20 11:20:34,355 - __main__ - INFO - raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] 2025-07-20 11:20:34,355 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:20:34,355 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:20:34,355 - sglang - INFO - urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='huggingface.co', port=443): Max retries exceeded with url: /api/models/allenai/olmOCR-7B-0225-preview (Caused by NewConnectionError(': Failed to establish a new connection: [Errno 101] Network is unreachable')) 2025-07-20 11:20:34,355 - __main__ - INFO - urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='huggingface.co', port=443): Max retries exceeded with url: /api/models/allenai/olmOCR-7B-0225-preview (Caused by NewConnectionError(': Failed to establish a new connection: [Errno 101] Network is unreachable')) 2025-07-20 11:20:34,355 - sglang - INFO - 2025-07-20 11:20:34,355 - __main__ - INFO - 2025-07-20 11:20:34,355 - sglang - INFO - During handling of the above exception, another exception occurred: 2025-07-20 11:20:34,355 - __main__ - INFO - During handling of the above exception, another exception occurred: 2025-07-20 11:20:34,356 - sglang - INFO - 2025-07-20 11:20:34,356 - __main__ - INFO - 2025-07-20 11:20:34,356 - sglang - INFO - Traceback (most recent call last): 2025-07-20 11:20:34,356 - __main__ - INFO - Traceback (most recent call last): 2025-07-20 11:20:34,356 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/managers/scheduler.py", line 1773, in run_scheduler_process 2025-07-20 11:20:34,356 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/managers/scheduler.py", line 1773, in run_scheduler_process 2025-07-20 11:20:34,356 - sglang - INFO - scheduler = Scheduler(server_args, port_args, gpu_id, tp_rank, dp_rank) 2025-07-20 11:20:34,356 - __main__ - INFO - scheduler = Scheduler(server_args, port_args, gpu_id, tp_rank, dp_rank) 2025-07-20 11:20:34,356 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:20:34,356 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:20:34,356 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/managers/scheduler.py", line 239, in __init__ 2025-07-20 11:20:34,356 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/managers/scheduler.py", line 239, in __init__ 2025-07-20 11:20:34,356 - sglang - INFO - self.tp_worker = TpWorkerClass( 2025-07-20 11:20:34,357 - __main__ - INFO - self.tp_worker = TpWorkerClass( 2025-07-20 11:20:34,357 - sglang - INFO - ^^^^^^^^^^^^^^ 2025-07-20 11:20:34,357 - __main__ - INFO - ^^^^^^^^^^^^^^ 2025-07-20 11:20:34,357 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/managers/tp_worker.py", line 68, in __init__ 2025-07-20 11:20:34,357 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/managers/tp_worker.py", line 68, in __init__ 2025-07-20 11:20:34,357 - sglang - INFO - self.model_runner = ModelRunner( 2025-07-20 11:20:34,357 - __main__ - INFO - self.model_runner = ModelRunner( 2025-07-20 11:20:34,357 - sglang - INFO - ^^^^^^^^^^^^ 2025-07-20 11:20:34,357 - __main__ - INFO - ^^^^^^^^^^^^ 2025-07-20 11:20:34,357 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/model_executor/model_runner.py", line 185, in __init__ 2025-07-20 11:20:34,357 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/model_executor/model_runner.py", line 185, in __init__ 2025-07-20 11:20:34,357 - sglang - INFO - self.load_model() 2025-07-20 11:20:34,357 - __main__ - INFO - self.load_model() 2025-07-20 11:20:34,358 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/model_executor/model_runner.py", line 306, in load_model 2025-07-20 11:20:34,358 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/model_executor/model_runner.py", line 306, in load_model 2025-07-20 11:20:34,358 - sglang - INFO - self.model = get_model( 2025-07-20 11:20:34,358 - __main__ - INFO - self.model = get_model( 2025-07-20 11:20:34,358 - sglang - INFO - ^^^^^^^^^^ 2025-07-20 11:20:34,358 - __main__ - INFO - ^^^^^^^^^^ 2025-07-20 11:20:34,358 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/model_loader/__init__.py", line 22, in get_model 2025-07-20 11:20:34,358 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/model_loader/__init__.py", line 22, in get_model 2025-07-20 11:20:34,358 - sglang - INFO - return loader.load_model( 2025-07-20 11:20:34,358 - __main__ - INFO - return loader.load_model( 2025-07-20 11:20:34,358 - sglang - INFO - ^^^^^^^^^^^^^^^^^^ 2025-07-20 11:20:34,358 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^ 2025-07-20 11:20:34,358 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/model_loader/loader.py", line 362, in load_model 2025-07-20 11:20:34,358 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/model_loader/loader.py", line 362, in load_model 2025-07-20 11:20:34,359 - sglang - INFO - model.load_weights(self._get_all_weights(model_config, model)) 2025-07-20 11:20:34,359 - __main__ - INFO - model.load_weights(self._get_all_weights(model_config, model)) 2025-07-20 11:20:34,359 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/models/qwen2_vl.py", line 576, in load_weights 2025-07-20 11:20:34,359 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/models/qwen2_vl.py", line 576, in load_weights 2025-07-20 11:20:34,359 - sglang - INFO - for name, loaded_weight in weights: 2025-07-20 11:20:34,359 - __main__ - INFO - for name, loaded_weight in weights: 2025-07-20 11:20:34,359 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/model_loader/loader.py", line 335, in _get_all_weights 2025-07-20 11:20:34,359 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/model_loader/loader.py", line 335, in _get_all_weights 2025-07-20 11:20:34,359 - sglang - INFO - yield from self._get_weights_iterator(primary_weights) 2025-07-20 11:20:34,359 - __main__ - INFO - yield from self._get_weights_iterator(primary_weights) 2025-07-20 11:20:34,359 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:20:34,359 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:20:34,359 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/model_loader/loader.py", line 303, in _get_weights_iterator 2025-07-20 11:20:34,360 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/model_loader/loader.py", line 303, in _get_weights_iterator 2025-07-20 11:20:34,360 - sglang - INFO - hf_folder, hf_weights_files, use_safetensors = self._prepare_weights( 2025-07-20 11:20:34,360 - __main__ - INFO - hf_folder, hf_weights_files, use_safetensors = self._prepare_weights( 2025-07-20 11:20:34,360 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:20:34,360 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:20:34,360 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/model_loader/loader.py", line 255, in _prepare_weights 2025-07-20 11:20:34,360 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/model_loader/loader.py", line 255, in _prepare_weights 2025-07-20 11:20:34,360 - sglang - INFO - hf_folder = download_weights_from_hf( 2025-07-20 11:20:34,360 - __main__ - INFO - hf_folder = download_weights_from_hf( 2025-07-20 11:20:34,360 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:20:34,360 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:20:34,360 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/model_loader/weight_utils.py", line 246, in download_weights_from_hf 2025-07-20 11:20:34,360 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/model_loader/weight_utils.py", line 246, in download_weights_from_hf 2025-07-20 11:20:34,361 - sglang - INFO - file_list = fs.ls(model_name_or_path, detail=False, revision=revision) 2025-07-20 11:20:34,361 - __main__ - INFO - file_list = fs.ls(model_name_or_path, detail=False, revision=revision) 2025-07-20 11:20:34,361 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:20:34,361 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:20:34,361 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/huggingface_hub/hf_file_system.py", line 368, in ls 2025-07-20 11:20:34,361 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/huggingface_hub/hf_file_system.py", line 368, in ls 2025-07-20 11:20:34,361 - sglang - INFO - resolved_path = self.resolve_path(path, revision=revision) 2025-07-20 11:20:34,361 - __main__ - INFO - resolved_path = self.resolve_path(path, revision=revision) 2025-07-20 11:20:34,361 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:20:34,361 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:20:34,361 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/huggingface_hub/hf_file_system.py", line 209, in resolve_path 2025-07-20 11:20:34,361 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/huggingface_hub/hf_file_system.py", line 209, in resolve_path 2025-07-20 11:20:34,361 - sglang - INFO - repo_and_revision_exist, err = self._repo_and_revision_exist(repo_type, repo_id, revision) 2025-07-20 11:20:34,361 - __main__ - INFO - repo_and_revision_exist, err = self._repo_and_revision_exist(repo_type, repo_id, revision) 2025-07-20 11:20:34,361 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:20:34,361 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:20:34,361 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/huggingface_hub/hf_file_system.py", line 125, in _repo_and_revision_exist 2025-07-20 11:20:34,361 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/huggingface_hub/hf_file_system.py", line 125, in _repo_and_revision_exist 2025-07-20 11:20:34,361 - sglang - INFO - self._api.repo_info( 2025-07-20 11:20:34,362 - __main__ - INFO - self._api.repo_info( 2025-07-20 11:20:34,363 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/huggingface_hub/utils/_validators.py", line 114, in _inner_fn 2025-07-20 11:20:34,363 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/huggingface_hub/utils/_validators.py", line 114, in _inner_fn 2025-07-20 11:20:34,363 - sglang - INFO - return fn(*args, **kwargs) 2025-07-20 11:20:34,363 - __main__ - INFO - return fn(*args, **kwargs) 2025-07-20 11:20:34,363 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:20:34,363 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:20:34,363 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/huggingface_hub/hf_api.py", line 2816, in repo_info 2025-07-20 11:20:34,363 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/huggingface_hub/hf_api.py", line 2816, in repo_info 2025-07-20 11:20:34,363 - sglang - INFO - return method( 2025-07-20 11:20:34,363 - __main__ - INFO - return method( 2025-07-20 11:20:34,363 - sglang - INFO - ^^^^^^^ 2025-07-20 11:20:34,363 - __main__ - INFO - ^^^^^^^ 2025-07-20 11:20:34,363 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/huggingface_hub/utils/_validators.py", line 114, in _inner_fn 2025-07-20 11:20:34,363 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/huggingface_hub/utils/_validators.py", line 114, in _inner_fn 2025-07-20 11:20:34,363 - sglang - INFO - return fn(*args, **kwargs) 2025-07-20 11:20:34,363 - __main__ - INFO - return fn(*args, **kwargs) 2025-07-20 11:20:34,364 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:20:34,364 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:20:34,364 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/huggingface_hub/hf_api.py", line 2600, in model_info 2025-07-20 11:20:34,364 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/huggingface_hub/hf_api.py", line 2600, in model_info 2025-07-20 11:20:34,364 - sglang - INFO - r = get_session().get(path, headers=headers, timeout=timeout, params=params) 2025-07-20 11:20:34,364 - __main__ - INFO - r = get_session().get(path, headers=headers, timeout=timeout, params=params) 2025-07-20 11:20:34,364 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:20:34,364 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:20:34,364 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/requests/sessions.py", line 602, in get 2025-07-20 11:20:34,364 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/requests/sessions.py", line 602, in get 2025-07-20 11:20:34,364 - sglang - INFO - return self.request("GET", url, **kwargs) 2025-07-20 11:20:34,364 - __main__ - INFO - return self.request("GET", url, **kwargs) 2025-07-20 11:20:34,364 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:20:34,364 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:20:34,364 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/requests/sessions.py", line 589, in request 2025-07-20 11:20:34,364 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/requests/sessions.py", line 589, in request 2025-07-20 11:20:34,364 - sglang - INFO - resp = self.send(prep, **send_kwargs) 2025-07-20 11:20:34,364 - __main__ - INFO - resp = self.send(prep, **send_kwargs) 2025-07-20 11:20:34,364 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:20:34,364 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:20:34,364 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/requests/sessions.py", line 703, in send 2025-07-20 11:20:34,364 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/requests/sessions.py", line 703, in send 2025-07-20 11:20:34,364 - sglang - INFO - r = adapter.send(request, **kwargs) 2025-07-20 11:20:34,364 - __main__ - INFO - r = adapter.send(request, **kwargs) 2025-07-20 11:20:34,365 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:20:34,365 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:20:34,365 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/huggingface_hub/utils/_http.py", line 96, in send 2025-07-20 11:20:34,365 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/huggingface_hub/utils/_http.py", line 96, in send 2025-07-20 11:20:34,365 - sglang - INFO - return super().send(request, *args, **kwargs) 2025-07-20 11:20:34,365 - __main__ - INFO - return super().send(request, *args, **kwargs) 2025-07-20 11:20:34,365 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:20:34,365 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:20:34,365 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/requests/adapters.py", line 700, in send 2025-07-20 11:20:34,365 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/requests/adapters.py", line 700, in send 2025-07-20 11:20:34,365 - sglang - INFO - raise ConnectionError(e, request=request) 2025-07-20 11:20:34,365 - __main__ - INFO - raise ConnectionError(e, request=request) 2025-07-20 11:20:34,365 - sglang - INFO - requests.exceptions.ConnectionError: (MaxRetryError("HTTPSConnectionPool(host='huggingface.co', port=443): Max retries exceeded with url: /api/models/allenai/olmOCR-7B-0225-preview (Caused by NewConnectionError(': Failed to establish a new connection: [Errno 101] Network is unreachable'))"), '(Request ID: 6a081490-df5f-4ba3-bb52-3fc81355011d)') 2025-07-20 11:20:34,365 - __main__ - INFO - requests.exceptions.ConnectionError: (MaxRetryError("HTTPSConnectionPool(host='huggingface.co', port=443): Max retries exceeded with url: /api/models/allenai/olmOCR-7B-0225-preview (Caused by NewConnectionError(': Failed to establish a new connection: [Errno 101] Network is unreachable'))"), '(Request ID: 6a081490-df5f-4ba3-bb52-3fc81355011d)') 2025-07-20 11:20:34,365 - sglang - INFO - 2025-07-20 11:20:34,365 - __main__ - INFO - 2025-07-20 11:20:34,365 - sglang - INFO - [2025-07-20 11:20:34] Received sigquit from a child proces. It usually means the child failed. 2025-07-20 11:20:34,365 - __main__ - INFO - [2025-07-20 11:20:34] Received sigquit from a child proces. It usually means the child failed. 2025-07-20 11:20:34,533 - __main__ - WARNING - Attempt 72: Please wait for sglang server to become ready... 2025-07-20 11:20:34,619 - __main__ - WARNING - SGLang server task ended 2025-07-20 11:20:40,624 - __main__ - WARNING - Attempt 73: Please wait for sglang server to become ready... 2025-07-20 11:20:42,094 - sglang - INFO - [2025-07-20 11:20:42] server_args=ServerArgs(model_path='allenai/olmOCR-7B-0225-preview', tokenizer_path='allenai/olmOCR-7B-0225-preview', tokenizer_mode='auto', load_format='auto', trust_remote_code=False, dtype='auto', kv_cache_dtype='auto', quantization_param_path=None, quantization=None, context_length=None, device='cuda', served_model_name='allenai/olmOCR-7B-0225-preview', chat_template='qwen2-vl', is_embedding=False, revision=None, skip_tokenizer_init=False, host='127.0.0.1', port=30024, mem_fraction_static=0.8, max_running_requests=None, max_total_tokens=None, chunked_prefill_size=2048, max_prefill_tokens=16384, schedule_policy='lpm', schedule_conservativeness=1.0, cpu_offload_gb=0, prefill_only_one_req=False, tp_size=1, stream_interval=1, stream_output=False, random_seed=741775413, constrained_json_whitespace_pattern=None, watchdog_timeout=300, download_dir=None, base_gpu_id=0, log_level='info', log_level_http='warning', log_requests=False, show_time_cost=False, enable_metrics=False, decode_log_interval=40, api_key=None, file_storage_pth='sglang_storage', enable_cache_report=False, dp_size=1, load_balance_method='round_robin', ep_size=1, dist_init_addr=None, nnodes=1, node_rank=0, json_model_override_args='{}', lora_paths=None, max_loras_per_batch=8, attention_backend='flashinfer', sampling_backend='flashinfer', grammar_backend='outlines', speculative_draft_model_path=None, speculative_algorithm=None, speculative_num_steps=5, speculative_num_draft_tokens=64, speculative_eagle_topk=8, enable_double_sparsity=False, ds_channel_config_path=None, ds_heavy_channel_num=32, ds_heavy_token_num=256, ds_heavy_channel_type='qk', ds_sparse_decode_threshold=4096, disable_radix_cache=False, disable_jump_forward=False, disable_cuda_graph=False, disable_cuda_graph_padding=False, disable_outlines_disk_cache=False, disable_custom_all_reduce=False, disable_mla=False, disable_overlap_schedule=False, enable_mixed_chunk=False, enable_dp_attention=False, enable_ep_moe=False, enable_torch_compile=False, torch_compile_max_bs=32, cuda_graph_max_bs=8, cuda_graph_bs=None, torchao_config='', enable_nan_detection=False, enable_p2p_check=False, triton_attention_reduce_in_fp32=False, triton_attention_num_kv_splits=8, num_continuous_decode_steps=1, delete_ckpt_after_loading=False, enable_memory_saver=False, allow_auto_truncate=False, enable_custom_logit_processor=False, tool_call_parser=None) 2025-07-20 11:20:42,094 - __main__ - INFO - [2025-07-20 11:20:42] server_args=ServerArgs(model_path='allenai/olmOCR-7B-0225-preview', tokenizer_path='allenai/olmOCR-7B-0225-preview', tokenizer_mode='auto', load_format='auto', trust_remote_code=False, dtype='auto', kv_cache_dtype='auto', quantization_param_path=None, quantization=None, context_length=None, device='cuda', served_model_name='allenai/olmOCR-7B-0225-preview', chat_template='qwen2-vl', is_embedding=False, revision=None, skip_tokenizer_init=False, host='127.0.0.1', port=30024, mem_fraction_static=0.8, max_running_requests=None, max_total_tokens=None, chunked_prefill_size=2048, max_prefill_tokens=16384, schedule_policy='lpm', schedule_conservativeness=1.0, cpu_offload_gb=0, prefill_only_one_req=False, tp_size=1, stream_interval=1, stream_output=False, random_seed=741775413, constrained_json_whitespace_pattern=None, watchdog_timeout=300, download_dir=None, base_gpu_id=0, log_level='info', log_level_http='warning', log_requests=False, show_time_cost=False, enable_metrics=False, decode_log_interval=40, api_key=None, file_storage_pth='sglang_storage', enable_cache_report=False, dp_size=1, load_balance_method='round_robin', ep_size=1, dist_init_addr=None, nnodes=1, node_rank=0, json_model_override_args='{}', lora_paths=None, max_loras_per_batch=8, attention_backend='flashinfer', sampling_backend='flashinfer', grammar_backend='outlines', speculative_draft_model_path=None, speculative_algorithm=None, speculative_num_steps=5, speculative_num_draft_tokens=64, speculative_eagle_topk=8, enable_double_sparsity=False, ds_channel_config_path=None, ds_heavy_channel_num=32, ds_heavy_token_num=256, ds_heavy_channel_type='qk', ds_sparse_decode_threshold=4096, disable_radix_cache=False, disable_jump_forward=False, disable_cuda_graph=False, disable_cuda_graph_padding=False, disable_outlines_disk_cache=False, disable_custom_all_reduce=False, disable_mla=False, disable_overlap_schedule=False, enable_mixed_chunk=False, enable_dp_attention=False, enable_ep_moe=False, enable_torch_compile=False, torch_compile_max_bs=32, cuda_graph_max_bs=8, cuda_graph_bs=None, torchao_config='', enable_nan_detection=False, enable_p2p_check=False, triton_attention_reduce_in_fp32=False, triton_attention_num_kv_splits=8, num_continuous_decode_steps=1, delete_ckpt_after_loading=False, enable_memory_saver=False, allow_auto_truncate=False, enable_custom_logit_processor=False, tool_call_parser=None) 2025-07-20 11:20:46,686 - __main__ - WARNING - Attempt 74: Please wait for sglang server to become ready... 2025-07-20 11:20:52,765 - __main__ - WARNING - Attempt 75: Please wait for sglang server to become ready... 2025-07-20 11:20:58,847 - __main__ - WARNING - Attempt 76: Please wait for sglang server to become ready... 2025-07-20 11:21:04,928 - __main__ - WARNING - Attempt 77: Please wait for sglang server to become ready... 2025-07-20 11:21:11,009 - __main__ - WARNING - Attempt 78: Please wait for sglang server to become ready... 2025-07-20 11:21:17,099 - __main__ - WARNING - Attempt 79: Please wait for sglang server to become ready... 2025-07-20 11:21:23,177 - __main__ - WARNING - Attempt 80: Please wait for sglang server to become ready... 2025-07-20 11:21:29,255 - __main__ - WARNING - Attempt 81: Please wait for sglang server to become ready... 2025-07-20 11:21:35,334 - __main__ - WARNING - Attempt 82: Please wait for sglang server to become ready... 2025-07-20 11:21:41,415 - __main__ - WARNING - Attempt 83: Please wait for sglang server to become ready... 2025-07-20 11:21:47,493 - __main__ - WARNING - Attempt 84: Please wait for sglang server to become ready... 2025-07-20 11:21:53,583 - __main__ - WARNING - Attempt 85: Please wait for sglang server to become ready... 2025-07-20 11:21:59,665 - __main__ - WARNING - Attempt 86: Please wait for sglang server to become ready... 2025-07-20 11:22:05,746 - __main__ - WARNING - Attempt 87: Please wait for sglang server to become ready... 2025-07-20 11:22:11,830 - __main__ - WARNING - Attempt 88: Please wait for sglang server to become ready... 2025-07-20 11:22:17,911 - __main__ - WARNING - Attempt 89: Please wait for sglang server to become ready... 2025-07-20 11:22:18,669 - sglang - INFO - [2025-07-20 11:22:18] Use chat template for the OpenAI-compatible API server: qwen2-vl 2025-07-20 11:22:18,670 - __main__ - INFO - [2025-07-20 11:22:18] Use chat template for the OpenAI-compatible API server: qwen2-vl 2025-07-20 11:22:23,992 - __main__ - WARNING - Attempt 90: Please wait for sglang server to become ready... 2025-07-20 11:22:24,827 - sglang - INFO - [2025-07-20 11:22:24 TP0] Overlap scheduler is disabled for multimodal models. 2025-07-20 11:22:24,827 - __main__ - INFO - [2025-07-20 11:22:24 TP0] Overlap scheduler is disabled for multimodal models. 2025-07-20 11:22:30,073 - __main__ - WARNING - Attempt 91: Please wait for sglang server to become ready... 2025-07-20 11:22:36,155 - __main__ - WARNING - Attempt 92: Please wait for sglang server to become ready... 2025-07-20 11:22:42,236 - __main__ - WARNING - Attempt 93: Please wait for sglang server to become ready... 2025-07-20 11:22:45,011 - sglang - INFO - [2025-07-20 11:22:45 TP0] Automatically reduce --mem-fraction-static to 0.760 because this is a multimodal model. 2025-07-20 11:22:45,012 - __main__ - INFO - [2025-07-20 11:22:45 TP0] Automatically reduce --mem-fraction-static to 0.760 because this is a multimodal model. 2025-07-20 11:22:45,012 - sglang - INFO - [2025-07-20 11:22:45 TP0] Automatically turn off --chunked-prefill-size and disable radix cache for qwen2-vl. 2025-07-20 11:22:45,012 - __main__ - INFO - [2025-07-20 11:22:45 TP0] Automatically turn off --chunked-prefill-size and disable radix cache for qwen2-vl. 2025-07-20 11:22:45,012 - sglang - INFO - [2025-07-20 11:22:45 TP0] Init torch distributed begin. 2025-07-20 11:22:45,012 - __main__ - INFO - [2025-07-20 11:22:45 TP0] Init torch distributed begin. 2025-07-20 11:22:48,322 - __main__ - WARNING - Attempt 94: Please wait for sglang server to become ready... 2025-07-20 11:22:50,393 - sglang - INFO - [2025-07-20 11:22:50 TP0] Load weight begin. avail mem=23.33 GB 2025-07-20 11:22:50,393 - __main__ - INFO - [2025-07-20 11:22:50 TP0] Load weight begin. avail mem=23.33 GB 2025-07-20 11:22:54,404 - __main__ - WARNING - Attempt 95: Please wait for sglang server to become ready... 2025-07-20 11:23:00,485 - __main__ - WARNING - Attempt 96: Please wait for sglang server to become ready... 2025-07-20 11:23:01,080 - sglang - INFO - [2025-07-20 11:23:01 TP0] Scheduler hit an exception: Traceback (most recent call last): 2025-07-20 11:23:01,080 - __main__ - INFO - [2025-07-20 11:23:01 TP0] Scheduler hit an exception: Traceback (most recent call last): 2025-07-20 11:23:01,080 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/urllib3/connection.py", line 198, in _new_conn 2025-07-20 11:23:01,080 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/urllib3/connection.py", line 198, in _new_conn 2025-07-20 11:23:01,080 - sglang - INFO - sock = connection.create_connection( 2025-07-20 11:23:01,080 - __main__ - INFO - sock = connection.create_connection( 2025-07-20 11:23:01,081 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:23:01,081 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:23:01,081 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/urllib3/util/connection.py", line 85, in create_connection 2025-07-20 11:23:01,081 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/urllib3/util/connection.py", line 85, in create_connection 2025-07-20 11:23:01,081 - sglang - INFO - raise err 2025-07-20 11:23:01,081 - __main__ - INFO - raise err 2025-07-20 11:23:01,081 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/urllib3/util/connection.py", line 73, in create_connection 2025-07-20 11:23:01,081 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/urllib3/util/connection.py", line 73, in create_connection 2025-07-20 11:23:01,081 - sglang - INFO - sock.connect(sa) 2025-07-20 11:23:01,081 - __main__ - INFO - sock.connect(sa) 2025-07-20 11:23:01,081 - sglang - INFO - OSError: [Errno 101] Network is unreachable 2025-07-20 11:23:01,081 - __main__ - INFO - OSError: [Errno 101] Network is unreachable 2025-07-20 11:23:01,081 - sglang - INFO - 2025-07-20 11:23:01,081 - __main__ - INFO - 2025-07-20 11:23:01,081 - sglang - INFO - The above exception was the direct cause of the following exception: 2025-07-20 11:23:01,081 - __main__ - INFO - The above exception was the direct cause of the following exception: 2025-07-20 11:23:01,081 - sglang - INFO - 2025-07-20 11:23:01,081 - __main__ - INFO - 2025-07-20 11:23:01,081 - sglang - INFO - Traceback (most recent call last): 2025-07-20 11:23:01,082 - __main__ - INFO - Traceback (most recent call last): 2025-07-20 11:23:01,082 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/urllib3/connectionpool.py", line 787, in urlopen 2025-07-20 11:23:01,082 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/urllib3/connectionpool.py", line 787, in urlopen 2025-07-20 11:23:01,082 - sglang - INFO - response = self._make_request( 2025-07-20 11:23:01,082 - __main__ - INFO - response = self._make_request( 2025-07-20 11:23:01,082 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:23:01,082 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:23:01,082 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/urllib3/connectionpool.py", line 488, in _make_request 2025-07-20 11:23:01,082 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/urllib3/connectionpool.py", line 488, in _make_request 2025-07-20 11:23:01,082 - sglang - INFO - raise new_e 2025-07-20 11:23:01,082 - __main__ - INFO - raise new_e 2025-07-20 11:23:01,082 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/urllib3/connectionpool.py", line 464, in _make_request 2025-07-20 11:23:01,083 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/urllib3/connectionpool.py", line 464, in _make_request 2025-07-20 11:23:01,083 - sglang - INFO - self._validate_conn(conn) 2025-07-20 11:23:01,083 - __main__ - INFO - self._validate_conn(conn) 2025-07-20 11:23:01,083 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/urllib3/connectionpool.py", line 1093, in _validate_conn 2025-07-20 11:23:01,083 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/urllib3/connectionpool.py", line 1093, in _validate_conn 2025-07-20 11:23:01,083 - sglang - INFO - conn.connect() 2025-07-20 11:23:01,083 - __main__ - INFO - conn.connect() 2025-07-20 11:23:01,083 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/urllib3/connection.py", line 704, in connect 2025-07-20 11:23:01,083 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/urllib3/connection.py", line 704, in connect 2025-07-20 11:23:01,083 - sglang - INFO - self.sock = sock = self._new_conn() 2025-07-20 11:23:01,083 - __main__ - INFO - self.sock = sock = self._new_conn() 2025-07-20 11:23:01,083 - sglang - INFO - ^^^^^^^^^^^^^^^^ 2025-07-20 11:23:01,083 - __main__ - INFO - ^^^^^^^^^^^^^^^^ 2025-07-20 11:23:01,084 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/urllib3/connection.py", line 213, in _new_conn 2025-07-20 11:23:01,084 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/urllib3/connection.py", line 213, in _new_conn 2025-07-20 11:23:01,084 - sglang - INFO - raise NewConnectionError( 2025-07-20 11:23:01,084 - __main__ - INFO - raise NewConnectionError( 2025-07-20 11:23:01,084 - sglang - INFO - urllib3.exceptions.NewConnectionError: : Failed to establish a new connection: [Errno 101] Network is unreachable 2025-07-20 11:23:01,084 - __main__ - INFO - urllib3.exceptions.NewConnectionError: : Failed to establish a new connection: [Errno 101] Network is unreachable 2025-07-20 11:23:01,084 - sglang - INFO - 2025-07-20 11:23:01,084 - __main__ - INFO - 2025-07-20 11:23:01,084 - sglang - INFO - The above exception was the direct cause of the following exception: 2025-07-20 11:23:01,084 - __main__ - INFO - The above exception was the direct cause of the following exception: 2025-07-20 11:23:01,084 - sglang - INFO - 2025-07-20 11:23:01,084 - __main__ - INFO - 2025-07-20 11:23:01,084 - sglang - INFO - Traceback (most recent call last): 2025-07-20 11:23:01,084 - __main__ - INFO - Traceback (most recent call last): 2025-07-20 11:23:01,084 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/requests/adapters.py", line 667, in send 2025-07-20 11:23:01,085 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/requests/adapters.py", line 667, in send 2025-07-20 11:23:01,085 - sglang - INFO - resp = conn.urlopen( 2025-07-20 11:23:01,085 - __main__ - INFO - resp = conn.urlopen( 2025-07-20 11:23:01,085 - sglang - INFO - ^^^^^^^^^^^^^ 2025-07-20 11:23:01,085 - __main__ - INFO - ^^^^^^^^^^^^^ 2025-07-20 11:23:01,085 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/urllib3/connectionpool.py", line 841, in urlopen 2025-07-20 11:23:01,085 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/urllib3/connectionpool.py", line 841, in urlopen 2025-07-20 11:23:01,085 - sglang - INFO - retries = retries.increment( 2025-07-20 11:23:01,085 - __main__ - INFO - retries = retries.increment( 2025-07-20 11:23:01,085 - sglang - INFO - ^^^^^^^^^^^^^^^^^^ 2025-07-20 11:23:01,085 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^ 2025-07-20 11:23:01,085 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/urllib3/util/retry.py", line 519, in increment 2025-07-20 11:23:01,085 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/urllib3/util/retry.py", line 519, in increment 2025-07-20 11:23:01,085 - sglang - INFO - raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] 2025-07-20 11:23:01,085 - __main__ - INFO - raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] 2025-07-20 11:23:01,085 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:23:01,085 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:23:01,085 - sglang - INFO - urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='huggingface.co', port=443): Max retries exceeded with url: /api/models/allenai/olmOCR-7B-0225-preview (Caused by NewConnectionError(': Failed to establish a new connection: [Errno 101] Network is unreachable')) 2025-07-20 11:23:01,085 - __main__ - INFO - urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='huggingface.co', port=443): Max retries exceeded with url: /api/models/allenai/olmOCR-7B-0225-preview (Caused by NewConnectionError(': Failed to establish a new connection: [Errno 101] Network is unreachable')) 2025-07-20 11:23:01,085 - sglang - INFO - 2025-07-20 11:23:01,085 - __main__ - INFO - 2025-07-20 11:23:01,085 - sglang - INFO - During handling of the above exception, another exception occurred: 2025-07-20 11:23:01,085 - __main__ - INFO - During handling of the above exception, another exception occurred: 2025-07-20 11:23:01,085 - sglang - INFO - 2025-07-20 11:23:01,085 - __main__ - INFO - 2025-07-20 11:23:01,085 - sglang - INFO - Traceback (most recent call last): 2025-07-20 11:23:01,085 - __main__ - INFO - Traceback (most recent call last): 2025-07-20 11:23:01,085 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/managers/scheduler.py", line 1773, in run_scheduler_process 2025-07-20 11:23:01,086 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/managers/scheduler.py", line 1773, in run_scheduler_process 2025-07-20 11:23:01,086 - sglang - INFO - scheduler = Scheduler(server_args, port_args, gpu_id, tp_rank, dp_rank) 2025-07-20 11:23:01,086 - __main__ - INFO - scheduler = Scheduler(server_args, port_args, gpu_id, tp_rank, dp_rank) 2025-07-20 11:23:01,086 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:23:01,086 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:23:01,086 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/managers/scheduler.py", line 239, in __init__ 2025-07-20 11:23:01,086 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/managers/scheduler.py", line 239, in __init__ 2025-07-20 11:23:01,086 - sglang - INFO - self.tp_worker = TpWorkerClass( 2025-07-20 11:23:01,086 - __main__ - INFO - self.tp_worker = TpWorkerClass( 2025-07-20 11:23:01,086 - sglang - INFO - ^^^^^^^^^^^^^^ 2025-07-20 11:23:01,086 - __main__ - INFO - ^^^^^^^^^^^^^^ 2025-07-20 11:23:01,086 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/managers/tp_worker.py", line 68, in __init__ 2025-07-20 11:23:01,086 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/managers/tp_worker.py", line 68, in __init__ 2025-07-20 11:23:01,086 - sglang - INFO - self.model_runner = ModelRunner( 2025-07-20 11:23:01,086 - __main__ - INFO - self.model_runner = ModelRunner( 2025-07-20 11:23:01,086 - sglang - INFO - ^^^^^^^^^^^^ 2025-07-20 11:23:01,086 - __main__ - INFO - ^^^^^^^^^^^^ 2025-07-20 11:23:01,086 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/model_executor/model_runner.py", line 185, in __init__ 2025-07-20 11:23:01,086 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/model_executor/model_runner.py", line 185, in __init__ 2025-07-20 11:23:01,086 - sglang - INFO - self.load_model() 2025-07-20 11:23:01,086 - __main__ - INFO - self.load_model() 2025-07-20 11:23:01,086 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/model_executor/model_runner.py", line 306, in load_model 2025-07-20 11:23:01,086 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/model_executor/model_runner.py", line 306, in load_model 2025-07-20 11:23:01,086 - sglang - INFO - self.model = get_model( 2025-07-20 11:23:01,087 - __main__ - INFO - self.model = get_model( 2025-07-20 11:23:01,087 - sglang - INFO - ^^^^^^^^^^ 2025-07-20 11:23:01,087 - __main__ - INFO - ^^^^^^^^^^ 2025-07-20 11:23:01,087 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/model_loader/__init__.py", line 22, in get_model 2025-07-20 11:23:01,087 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/model_loader/__init__.py", line 22, in get_model 2025-07-20 11:23:01,087 - sglang - INFO - return loader.load_model( 2025-07-20 11:23:01,087 - __main__ - INFO - return loader.load_model( 2025-07-20 11:23:01,087 - sglang - INFO - ^^^^^^^^^^^^^^^^^^ 2025-07-20 11:23:01,087 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^ 2025-07-20 11:23:01,087 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/model_loader/loader.py", line 362, in load_model 2025-07-20 11:23:01,087 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/model_loader/loader.py", line 362, in load_model 2025-07-20 11:23:01,087 - sglang - INFO - model.load_weights(self._get_all_weights(model_config, model)) 2025-07-20 11:23:01,087 - __main__ - INFO - model.load_weights(self._get_all_weights(model_config, model)) 2025-07-20 11:23:01,087 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/models/qwen2_vl.py", line 576, in load_weights 2025-07-20 11:23:01,087 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/models/qwen2_vl.py", line 576, in load_weights 2025-07-20 11:23:01,087 - sglang - INFO - for name, loaded_weight in weights: 2025-07-20 11:23:01,087 - __main__ - INFO - for name, loaded_weight in weights: 2025-07-20 11:23:01,087 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/model_loader/loader.py", line 335, in _get_all_weights 2025-07-20 11:23:01,087 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/model_loader/loader.py", line 335, in _get_all_weights 2025-07-20 11:23:01,087 - sglang - INFO - yield from self._get_weights_iterator(primary_weights) 2025-07-20 11:23:01,087 - __main__ - INFO - yield from self._get_weights_iterator(primary_weights) 2025-07-20 11:23:01,087 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:23:01,087 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:23:01,087 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/model_loader/loader.py", line 303, in _get_weights_iterator 2025-07-20 11:23:01,088 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/model_loader/loader.py", line 303, in _get_weights_iterator 2025-07-20 11:23:01,088 - sglang - INFO - hf_folder, hf_weights_files, use_safetensors = self._prepare_weights( 2025-07-20 11:23:01,088 - __main__ - INFO - hf_folder, hf_weights_files, use_safetensors = self._prepare_weights( 2025-07-20 11:23:01,088 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:23:01,088 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:23:01,088 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/model_loader/loader.py", line 255, in _prepare_weights 2025-07-20 11:23:01,088 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/model_loader/loader.py", line 255, in _prepare_weights 2025-07-20 11:23:01,088 - sglang - INFO - hf_folder = download_weights_from_hf( 2025-07-20 11:23:01,088 - __main__ - INFO - hf_folder = download_weights_from_hf( 2025-07-20 11:23:01,088 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:23:01,088 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:23:01,088 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/model_loader/weight_utils.py", line 246, in download_weights_from_hf 2025-07-20 11:23:01,088 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/model_loader/weight_utils.py", line 246, in download_weights_from_hf 2025-07-20 11:23:01,088 - sglang - INFO - file_list = fs.ls(model_name_or_path, detail=False, revision=revision) 2025-07-20 11:23:01,088 - __main__ - INFO - file_list = fs.ls(model_name_or_path, detail=False, revision=revision) 2025-07-20 11:23:01,088 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:23:01,088 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:23:01,088 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/huggingface_hub/hf_file_system.py", line 368, in ls 2025-07-20 11:23:01,088 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/huggingface_hub/hf_file_system.py", line 368, in ls 2025-07-20 11:23:01,088 - sglang - INFO - resolved_path = self.resolve_path(path, revision=revision) 2025-07-20 11:23:01,088 - __main__ - INFO - resolved_path = self.resolve_path(path, revision=revision) 2025-07-20 11:23:01,088 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:23:01,088 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:23:01,089 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/huggingface_hub/hf_file_system.py", line 209, in resolve_path 2025-07-20 11:23:01,089 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/huggingface_hub/hf_file_system.py", line 209, in resolve_path 2025-07-20 11:23:01,089 - sglang - INFO - repo_and_revision_exist, err = self._repo_and_revision_exist(repo_type, repo_id, revision) 2025-07-20 11:23:01,089 - __main__ - INFO - repo_and_revision_exist, err = self._repo_and_revision_exist(repo_type, repo_id, revision) 2025-07-20 11:23:01,089 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:23:01,089 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:23:01,089 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/huggingface_hub/hf_file_system.py", line 125, in _repo_and_revision_exist 2025-07-20 11:23:01,089 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/huggingface_hub/hf_file_system.py", line 125, in _repo_and_revision_exist 2025-07-20 11:23:01,089 - sglang - INFO - self._api.repo_info( 2025-07-20 11:23:01,089 - __main__ - INFO - self._api.repo_info( 2025-07-20 11:23:01,089 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/huggingface_hub/utils/_validators.py", line 114, in _inner_fn 2025-07-20 11:23:01,089 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/huggingface_hub/utils/_validators.py", line 114, in _inner_fn 2025-07-20 11:23:01,089 - sglang - INFO - return fn(*args, **kwargs) 2025-07-20 11:23:01,089 - __main__ - INFO - return fn(*args, **kwargs) 2025-07-20 11:23:01,089 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:23:01,089 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:23:01,089 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/huggingface_hub/hf_api.py", line 2816, in repo_info 2025-07-20 11:23:01,089 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/huggingface_hub/hf_api.py", line 2816, in repo_info 2025-07-20 11:23:01,089 - sglang - INFO - return method( 2025-07-20 11:23:01,089 - __main__ - INFO - return method( 2025-07-20 11:23:01,089 - sglang - INFO - ^^^^^^^ 2025-07-20 11:23:01,089 - __main__ - INFO - ^^^^^^^ 2025-07-20 11:23:01,089 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/huggingface_hub/utils/_validators.py", line 114, in _inner_fn 2025-07-20 11:23:01,089 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/huggingface_hub/utils/_validators.py", line 114, in _inner_fn 2025-07-20 11:23:01,090 - sglang - INFO - return fn(*args, **kwargs) 2025-07-20 11:23:01,090 - __main__ - INFO - return fn(*args, **kwargs) 2025-07-20 11:23:01,090 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:23:01,090 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:23:01,090 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/huggingface_hub/hf_api.py", line 2600, in model_info 2025-07-20 11:23:01,090 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/huggingface_hub/hf_api.py", line 2600, in model_info 2025-07-20 11:23:01,090 - sglang - INFO - r = get_session().get(path, headers=headers, timeout=timeout, params=params) 2025-07-20 11:23:01,090 - __main__ - INFO - r = get_session().get(path, headers=headers, timeout=timeout, params=params) 2025-07-20 11:23:01,090 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:23:01,090 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:23:01,090 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/requests/sessions.py", line 602, in get 2025-07-20 11:23:01,090 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/requests/sessions.py", line 602, in get 2025-07-20 11:23:01,090 - sglang - INFO - return self.request("GET", url, **kwargs) 2025-07-20 11:23:01,090 - __main__ - INFO - return self.request("GET", url, **kwargs) 2025-07-20 11:23:01,090 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:23:01,090 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:23:01,090 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/requests/sessions.py", line 589, in request 2025-07-20 11:23:01,090 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/requests/sessions.py", line 589, in request 2025-07-20 11:23:01,090 - sglang - INFO - resp = self.send(prep, **send_kwargs) 2025-07-20 11:23:01,090 - __main__ - INFO - resp = self.send(prep, **send_kwargs) 2025-07-20 11:23:01,090 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:23:01,090 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:23:01,090 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/requests/sessions.py", line 703, in send 2025-07-20 11:23:01,090 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/requests/sessions.py", line 703, in send 2025-07-20 11:23:01,091 - sglang - INFO - r = adapter.send(request, **kwargs) 2025-07-20 11:23:01,091 - __main__ - INFO - r = adapter.send(request, **kwargs) 2025-07-20 11:23:01,091 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:23:01,091 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:23:01,091 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/huggingface_hub/utils/_http.py", line 96, in send 2025-07-20 11:23:01,091 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/huggingface_hub/utils/_http.py", line 96, in send 2025-07-20 11:23:01,091 - sglang - INFO - return super().send(request, *args, **kwargs) 2025-07-20 11:23:01,091 - __main__ - INFO - return super().send(request, *args, **kwargs) 2025-07-20 11:23:01,091 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:23:01,091 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:23:01,091 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/requests/adapters.py", line 700, in send 2025-07-20 11:23:01,091 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/requests/adapters.py", line 700, in send 2025-07-20 11:23:01,091 - sglang - INFO - raise ConnectionError(e, request=request) 2025-07-20 11:23:01,091 - __main__ - INFO - raise ConnectionError(e, request=request) 2025-07-20 11:23:01,091 - sglang - INFO - requests.exceptions.ConnectionError: (MaxRetryError("HTTPSConnectionPool(host='huggingface.co', port=443): Max retries exceeded with url: /api/models/allenai/olmOCR-7B-0225-preview (Caused by NewConnectionError(': Failed to establish a new connection: [Errno 101] Network is unreachable'))"), '(Request ID: 3a71c81f-a953-43be-9246-0327729c923d)') 2025-07-20 11:23:01,091 - __main__ - INFO - requests.exceptions.ConnectionError: (MaxRetryError("HTTPSConnectionPool(host='huggingface.co', port=443): Max retries exceeded with url: /api/models/allenai/olmOCR-7B-0225-preview (Caused by NewConnectionError(': Failed to establish a new connection: [Errno 101] Network is unreachable'))"), '(Request ID: 3a71c81f-a953-43be-9246-0327729c923d)') 2025-07-20 11:23:01,091 - sglang - INFO - 2025-07-20 11:23:01,091 - __main__ - INFO - 2025-07-20 11:23:01,091 - sglang - INFO - [2025-07-20 11:23:01] Received sigquit from a child proces. It usually means the child failed. 2025-07-20 11:23:01,091 - __main__ - INFO - [2025-07-20 11:23:01] Received sigquit from a child proces. It usually means the child failed. 2025-07-20 11:23:01,432 - __main__ - WARNING - SGLang server task ended 2025-07-20 11:23:06,644 - __main__ - WARNING - Attempt 97: Please wait for sglang server to become ready... 2025-07-20 11:23:08,746 - sglang - INFO - [2025-07-20 11:23:08] server_args=ServerArgs(model_path='allenai/olmOCR-7B-0225-preview', tokenizer_path='allenai/olmOCR-7B-0225-preview', tokenizer_mode='auto', load_format='auto', trust_remote_code=False, dtype='auto', kv_cache_dtype='auto', quantization_param_path=None, quantization=None, context_length=None, device='cuda', served_model_name='allenai/olmOCR-7B-0225-preview', chat_template='qwen2-vl', is_embedding=False, revision=None, skip_tokenizer_init=False, host='127.0.0.1', port=30024, mem_fraction_static=0.8, max_running_requests=None, max_total_tokens=None, chunked_prefill_size=2048, max_prefill_tokens=16384, schedule_policy='lpm', schedule_conservativeness=1.0, cpu_offload_gb=0, prefill_only_one_req=False, tp_size=1, stream_interval=1, stream_output=False, random_seed=935034446, constrained_json_whitespace_pattern=None, watchdog_timeout=300, download_dir=None, base_gpu_id=0, log_level='info', log_level_http='warning', log_requests=False, show_time_cost=False, enable_metrics=False, decode_log_interval=40, api_key=None, file_storage_pth='sglang_storage', enable_cache_report=False, dp_size=1, load_balance_method='round_robin', ep_size=1, dist_init_addr=None, nnodes=1, node_rank=0, json_model_override_args='{}', lora_paths=None, max_loras_per_batch=8, attention_backend='flashinfer', sampling_backend='flashinfer', grammar_backend='outlines', speculative_draft_model_path=None, speculative_algorithm=None, speculative_num_steps=5, speculative_num_draft_tokens=64, speculative_eagle_topk=8, enable_double_sparsity=False, ds_channel_config_path=None, ds_heavy_channel_num=32, ds_heavy_token_num=256, ds_heavy_channel_type='qk', ds_sparse_decode_threshold=4096, disable_radix_cache=False, disable_jump_forward=False, disable_cuda_graph=False, disable_cuda_graph_padding=False, disable_outlines_disk_cache=False, disable_custom_all_reduce=False, disable_mla=False, disable_overlap_schedule=False, enable_mixed_chunk=False, enable_dp_attention=False, enable_ep_moe=False, enable_torch_compile=False, torch_compile_max_bs=32, cuda_graph_max_bs=8, cuda_graph_bs=None, torchao_config='', enable_nan_detection=False, enable_p2p_check=False, triton_attention_reduce_in_fp32=False, triton_attention_num_kv_splits=8, num_continuous_decode_steps=1, delete_ckpt_after_loading=False, enable_memory_saver=False, allow_auto_truncate=False, enable_custom_logit_processor=False, tool_call_parser=None) 2025-07-20 11:23:08,746 - __main__ - INFO - [2025-07-20 11:23:08] server_args=ServerArgs(model_path='allenai/olmOCR-7B-0225-preview', tokenizer_path='allenai/olmOCR-7B-0225-preview', tokenizer_mode='auto', load_format='auto', trust_remote_code=False, dtype='auto', kv_cache_dtype='auto', quantization_param_path=None, quantization=None, context_length=None, device='cuda', served_model_name='allenai/olmOCR-7B-0225-preview', chat_template='qwen2-vl', is_embedding=False, revision=None, skip_tokenizer_init=False, host='127.0.0.1', port=30024, mem_fraction_static=0.8, max_running_requests=None, max_total_tokens=None, chunked_prefill_size=2048, max_prefill_tokens=16384, schedule_policy='lpm', schedule_conservativeness=1.0, cpu_offload_gb=0, prefill_only_one_req=False, tp_size=1, stream_interval=1, stream_output=False, random_seed=935034446, constrained_json_whitespace_pattern=None, watchdog_timeout=300, download_dir=None, base_gpu_id=0, log_level='info', log_level_http='warning', log_requests=False, show_time_cost=False, enable_metrics=False, decode_log_interval=40, api_key=None, file_storage_pth='sglang_storage', enable_cache_report=False, dp_size=1, load_balance_method='round_robin', ep_size=1, dist_init_addr=None, nnodes=1, node_rank=0, json_model_override_args='{}', lora_paths=None, max_loras_per_batch=8, attention_backend='flashinfer', sampling_backend='flashinfer', grammar_backend='outlines', speculative_draft_model_path=None, speculative_algorithm=None, speculative_num_steps=5, speculative_num_draft_tokens=64, speculative_eagle_topk=8, enable_double_sparsity=False, ds_channel_config_path=None, ds_heavy_channel_num=32, ds_heavy_token_num=256, ds_heavy_channel_type='qk', ds_sparse_decode_threshold=4096, disable_radix_cache=False, disable_jump_forward=False, disable_cuda_graph=False, disable_cuda_graph_padding=False, disable_outlines_disk_cache=False, disable_custom_all_reduce=False, disable_mla=False, disable_overlap_schedule=False, enable_mixed_chunk=False, enable_dp_attention=False, enable_ep_moe=False, enable_torch_compile=False, torch_compile_max_bs=32, cuda_graph_max_bs=8, cuda_graph_bs=None, torchao_config='', enable_nan_detection=False, enable_p2p_check=False, triton_attention_reduce_in_fp32=False, triton_attention_num_kv_splits=8, num_continuous_decode_steps=1, delete_ckpt_after_loading=False, enable_memory_saver=False, allow_auto_truncate=False, enable_custom_logit_processor=False, tool_call_parser=None) 2025-07-20 11:23:12,705 - __main__ - WARNING - Attempt 98: Please wait for sglang server to become ready... 2025-07-20 11:23:18,766 - __main__ - WARNING - Attempt 99: Please wait for sglang server to become ready... 2025-07-20 11:23:24,846 - __main__ - WARNING - Attempt 100: Please wait for sglang server to become ready... 2025-07-20 11:23:30,927 - __main__ - WARNING - Attempt 101: Please wait for sglang server to become ready... 2025-07-20 11:23:37,008 - __main__ - WARNING - Attempt 102: Please wait for sglang server to become ready... 2025-07-20 11:23:43,087 - __main__ - WARNING - Attempt 103: Please wait for sglang server to become ready... 2025-07-20 11:23:49,168 - __main__ - WARNING - Attempt 104: Please wait for sglang server to become ready... 2025-07-20 11:23:55,248 - __main__ - WARNING - Attempt 105: Please wait for sglang server to become ready... 2025-07-20 11:24:01,341 - __main__ - WARNING - Attempt 106: Please wait for sglang server to become ready... 2025-07-20 11:24:07,423 - __main__ - WARNING - Attempt 107: Please wait for sglang server to become ready... 2025-07-20 11:24:13,505 - __main__ - WARNING - Attempt 108: Please wait for sglang server to become ready... 2025-07-20 11:24:19,585 - __main__ - WARNING - Attempt 109: Please wait for sglang server to become ready... 2025-07-20 11:24:25,666 - __main__ - WARNING - Attempt 110: Please wait for sglang server to become ready... 2025-07-20 11:24:31,748 - __main__ - WARNING - Attempt 111: Please wait for sglang server to become ready... 2025-07-20 11:24:37,829 - __main__ - WARNING - Attempt 112: Please wait for sglang server to become ready... 2025-07-20 11:24:43,910 - __main__ - WARNING - Attempt 113: Please wait for sglang server to become ready... 2025-07-20 11:24:45,439 - sglang - INFO - [2025-07-20 11:24:45] Use chat template for the OpenAI-compatible API server: qwen2-vl 2025-07-20 11:24:45,439 - __main__ - INFO - [2025-07-20 11:24:45] Use chat template for the OpenAI-compatible API server: qwen2-vl 2025-07-20 11:24:49,991 - __main__ - WARNING - Attempt 114: Please wait for sglang server to become ready... 2025-07-20 11:24:51,481 - sglang - INFO - [2025-07-20 11:24:51 TP0] Overlap scheduler is disabled for multimodal models. 2025-07-20 11:24:51,481 - __main__ - INFO - [2025-07-20 11:24:51 TP0] Overlap scheduler is disabled for multimodal models. 2025-07-20 11:24:56,075 - __main__ - WARNING - Attempt 115: Please wait for sglang server to become ready... 2025-07-20 11:25:02,156 - __main__ - WARNING - Attempt 116: Please wait for sglang server to become ready... 2025-07-20 11:25:08,237 - __main__ - WARNING - Attempt 117: Please wait for sglang server to become ready... 2025-07-20 11:25:11,652 - sglang - INFO - [2025-07-20 11:25:11 TP0] Automatically reduce --mem-fraction-static to 0.760 because this is a multimodal model. 2025-07-20 11:25:11,653 - __main__ - INFO - [2025-07-20 11:25:11 TP0] Automatically reduce --mem-fraction-static to 0.760 because this is a multimodal model. 2025-07-20 11:25:11,653 - sglang - INFO - [2025-07-20 11:25:11 TP0] Automatically turn off --chunked-prefill-size and disable radix cache for qwen2-vl. 2025-07-20 11:25:11,653 - __main__ - INFO - [2025-07-20 11:25:11 TP0] Automatically turn off --chunked-prefill-size and disable radix cache for qwen2-vl. 2025-07-20 11:25:11,653 - sglang - INFO - [2025-07-20 11:25:11 TP0] Init torch distributed begin. 2025-07-20 11:25:11,653 - __main__ - INFO - [2025-07-20 11:25:11 TP0] Init torch distributed begin. 2025-07-20 11:25:14,319 - __main__ - WARNING - Attempt 118: Please wait for sglang server to become ready... 2025-07-20 11:25:17,053 - sglang - INFO - [2025-07-20 11:25:17 TP0] Load weight begin. avail mem=23.33 GB 2025-07-20 11:25:17,053 - __main__ - INFO - [2025-07-20 11:25:17 TP0] Load weight begin. avail mem=23.33 GB 2025-07-20 11:25:20,401 - __main__ - WARNING - Attempt 119: Please wait for sglang server to become ready... 2025-07-20 11:25:26,481 - __main__ - WARNING - Attempt 120: Please wait for sglang server to become ready... 2025-07-20 11:25:27,747 - sglang - INFO - [2025-07-20 11:25:27 TP0] Scheduler hit an exception: Traceback (most recent call last): 2025-07-20 11:25:27,748 - __main__ - INFO - [2025-07-20 11:25:27 TP0] Scheduler hit an exception: Traceback (most recent call last): 2025-07-20 11:25:27,748 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/urllib3/connection.py", line 198, in _new_conn 2025-07-20 11:25:27,748 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/urllib3/connection.py", line 198, in _new_conn 2025-07-20 11:25:27,748 - sglang - INFO - sock = connection.create_connection( 2025-07-20 11:25:27,748 - __main__ - INFO - sock = connection.create_connection( 2025-07-20 11:25:27,748 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:25:27,748 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:25:27,748 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/urllib3/util/connection.py", line 85, in create_connection 2025-07-20 11:25:27,748 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/urllib3/util/connection.py", line 85, in create_connection 2025-07-20 11:25:27,748 - sglang - INFO - raise err 2025-07-20 11:25:27,748 - __main__ - INFO - raise err 2025-07-20 11:25:27,748 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/urllib3/util/connection.py", line 73, in create_connection 2025-07-20 11:25:27,748 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/urllib3/util/connection.py", line 73, in create_connection 2025-07-20 11:25:27,748 - sglang - INFO - sock.connect(sa) 2025-07-20 11:25:27,748 - __main__ - INFO - sock.connect(sa) 2025-07-20 11:25:27,748 - sglang - INFO - OSError: [Errno 101] Network is unreachable 2025-07-20 11:25:27,748 - __main__ - INFO - OSError: [Errno 101] Network is unreachable 2025-07-20 11:25:27,749 - sglang - INFO - 2025-07-20 11:25:27,749 - __main__ - INFO - 2025-07-20 11:25:27,749 - sglang - INFO - The above exception was the direct cause of the following exception: 2025-07-20 11:25:27,749 - __main__ - INFO - The above exception was the direct cause of the following exception: 2025-07-20 11:25:27,749 - sglang - INFO - 2025-07-20 11:25:27,749 - __main__ - INFO - 2025-07-20 11:25:27,749 - sglang - INFO - Traceback (most recent call last): 2025-07-20 11:25:27,749 - __main__ - INFO - Traceback (most recent call last): 2025-07-20 11:25:27,749 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/urllib3/connectionpool.py", line 787, in urlopen 2025-07-20 11:25:27,749 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/urllib3/connectionpool.py", line 787, in urlopen 2025-07-20 11:25:27,749 - sglang - INFO - response = self._make_request( 2025-07-20 11:25:27,749 - __main__ - INFO - response = self._make_request( 2025-07-20 11:25:27,749 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:25:27,749 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:25:27,749 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/urllib3/connectionpool.py", line 488, in _make_request 2025-07-20 11:25:27,749 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/urllib3/connectionpool.py", line 488, in _make_request 2025-07-20 11:25:27,749 - sglang - INFO - raise new_e 2025-07-20 11:25:27,749 - __main__ - INFO - raise new_e 2025-07-20 11:25:27,749 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/urllib3/connectionpool.py", line 464, in _make_request 2025-07-20 11:25:27,749 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/urllib3/connectionpool.py", line 464, in _make_request 2025-07-20 11:25:27,749 - sglang - INFO - self._validate_conn(conn) 2025-07-20 11:25:27,749 - __main__ - INFO - self._validate_conn(conn) 2025-07-20 11:25:27,749 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/urllib3/connectionpool.py", line 1093, in _validate_conn 2025-07-20 11:25:27,750 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/urllib3/connectionpool.py", line 1093, in _validate_conn 2025-07-20 11:25:27,750 - sglang - INFO - conn.connect() 2025-07-20 11:25:27,750 - __main__ - INFO - conn.connect() 2025-07-20 11:25:27,750 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/urllib3/connection.py", line 704, in connect 2025-07-20 11:25:27,750 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/urllib3/connection.py", line 704, in connect 2025-07-20 11:25:27,750 - sglang - INFO - self.sock = sock = self._new_conn() 2025-07-20 11:25:27,750 - __main__ - INFO - self.sock = sock = self._new_conn() 2025-07-20 11:25:27,750 - sglang - INFO - ^^^^^^^^^^^^^^^^ 2025-07-20 11:25:27,750 - __main__ - INFO - ^^^^^^^^^^^^^^^^ 2025-07-20 11:25:27,750 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/urllib3/connection.py", line 213, in _new_conn 2025-07-20 11:25:27,750 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/urllib3/connection.py", line 213, in _new_conn 2025-07-20 11:25:27,750 - sglang - INFO - raise NewConnectionError( 2025-07-20 11:25:27,750 - __main__ - INFO - raise NewConnectionError( 2025-07-20 11:25:27,750 - sglang - INFO - urllib3.exceptions.NewConnectionError: : Failed to establish a new connection: [Errno 101] Network is unreachable 2025-07-20 11:25:27,750 - __main__ - INFO - urllib3.exceptions.NewConnectionError: : Failed to establish a new connection: [Errno 101] Network is unreachable 2025-07-20 11:25:27,750 - sglang - INFO - 2025-07-20 11:25:27,750 - __main__ - INFO - 2025-07-20 11:25:27,750 - sglang - INFO - The above exception was the direct cause of the following exception: 2025-07-20 11:25:27,750 - __main__ - INFO - The above exception was the direct cause of the following exception: 2025-07-20 11:25:27,750 - sglang - INFO - 2025-07-20 11:25:27,750 - __main__ - INFO - 2025-07-20 11:25:27,750 - sglang - INFO - Traceback (most recent call last): 2025-07-20 11:25:27,750 - __main__ - INFO - Traceback (most recent call last): 2025-07-20 11:25:27,750 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/requests/adapters.py", line 667, in send 2025-07-20 11:25:27,751 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/requests/adapters.py", line 667, in send 2025-07-20 11:25:27,751 - sglang - INFO - resp = conn.urlopen( 2025-07-20 11:25:27,751 - __main__ - INFO - resp = conn.urlopen( 2025-07-20 11:25:27,751 - sglang - INFO - ^^^^^^^^^^^^^ 2025-07-20 11:25:27,751 - __main__ - INFO - ^^^^^^^^^^^^^ 2025-07-20 11:25:27,751 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/urllib3/connectionpool.py", line 841, in urlopen 2025-07-20 11:25:27,751 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/urllib3/connectionpool.py", line 841, in urlopen 2025-07-20 11:25:27,751 - sglang - INFO - retries = retries.increment( 2025-07-20 11:25:27,751 - __main__ - INFO - retries = retries.increment( 2025-07-20 11:25:27,751 - sglang - INFO - ^^^^^^^^^^^^^^^^^^ 2025-07-20 11:25:27,751 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^ 2025-07-20 11:25:27,751 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/urllib3/util/retry.py", line 519, in increment 2025-07-20 11:25:27,751 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/urllib3/util/retry.py", line 519, in increment 2025-07-20 11:25:27,751 - sglang - INFO - raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] 2025-07-20 11:25:27,751 - __main__ - INFO - raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] 2025-07-20 11:25:27,751 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:25:27,751 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:25:27,751 - sglang - INFO - urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='huggingface.co', port=443): Max retries exceeded with url: /api/models/allenai/olmOCR-7B-0225-preview (Caused by NewConnectionError(': Failed to establish a new connection: [Errno 101] Network is unreachable')) 2025-07-20 11:25:27,751 - __main__ - INFO - urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='huggingface.co', port=443): Max retries exceeded with url: /api/models/allenai/olmOCR-7B-0225-preview (Caused by NewConnectionError(': Failed to establish a new connection: [Errno 101] Network is unreachable')) 2025-07-20 11:25:27,751 - sglang - INFO - 2025-07-20 11:25:27,751 - __main__ - INFO - 2025-07-20 11:25:27,751 - sglang - INFO - During handling of the above exception, another exception occurred: 2025-07-20 11:25:27,752 - __main__ - INFO - During handling of the above exception, another exception occurred: 2025-07-20 11:25:27,752 - sglang - INFO - 2025-07-20 11:25:27,752 - __main__ - INFO - 2025-07-20 11:25:27,752 - sglang - INFO - Traceback (most recent call last): 2025-07-20 11:25:27,752 - __main__ - INFO - Traceback (most recent call last): 2025-07-20 11:25:27,752 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/managers/scheduler.py", line 1773, in run_scheduler_process 2025-07-20 11:25:27,752 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/managers/scheduler.py", line 1773, in run_scheduler_process 2025-07-20 11:25:27,752 - sglang - INFO - scheduler = Scheduler(server_args, port_args, gpu_id, tp_rank, dp_rank) 2025-07-20 11:25:27,752 - __main__ - INFO - scheduler = Scheduler(server_args, port_args, gpu_id, tp_rank, dp_rank) 2025-07-20 11:25:27,752 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:25:27,752 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:25:27,752 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/managers/scheduler.py", line 239, in __init__ 2025-07-20 11:25:27,752 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/managers/scheduler.py", line 239, in __init__ 2025-07-20 11:25:27,752 - sglang - INFO - self.tp_worker = TpWorkerClass( 2025-07-20 11:25:27,752 - __main__ - INFO - self.tp_worker = TpWorkerClass( 2025-07-20 11:25:27,752 - sglang - INFO - ^^^^^^^^^^^^^^ 2025-07-20 11:25:27,752 - __main__ - INFO - ^^^^^^^^^^^^^^ 2025-07-20 11:25:27,752 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/managers/tp_worker.py", line 68, in __init__ 2025-07-20 11:25:27,752 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/managers/tp_worker.py", line 68, in __init__ 2025-07-20 11:25:27,752 - sglang - INFO - self.model_runner = ModelRunner( 2025-07-20 11:25:27,752 - __main__ - INFO - self.model_runner = ModelRunner( 2025-07-20 11:25:27,752 - sglang - INFO - ^^^^^^^^^^^^ 2025-07-20 11:25:27,752 - __main__ - INFO - ^^^^^^^^^^^^ 2025-07-20 11:25:27,753 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/model_executor/model_runner.py", line 185, in __init__ 2025-07-20 11:25:27,753 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/model_executor/model_runner.py", line 185, in __init__ 2025-07-20 11:25:27,753 - sglang - INFO - self.load_model() 2025-07-20 11:25:27,753 - __main__ - INFO - self.load_model() 2025-07-20 11:25:27,753 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/model_executor/model_runner.py", line 306, in load_model 2025-07-20 11:25:27,753 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/model_executor/model_runner.py", line 306, in load_model 2025-07-20 11:25:27,753 - sglang - INFO - self.model = get_model( 2025-07-20 11:25:27,753 - __main__ - INFO - self.model = get_model( 2025-07-20 11:25:27,753 - sglang - INFO - ^^^^^^^^^^ 2025-07-20 11:25:27,753 - __main__ - INFO - ^^^^^^^^^^ 2025-07-20 11:25:27,753 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/model_loader/__init__.py", line 22, in get_model 2025-07-20 11:25:27,753 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/model_loader/__init__.py", line 22, in get_model 2025-07-20 11:25:27,753 - sglang - INFO - return loader.load_model( 2025-07-20 11:25:27,753 - __main__ - INFO - return loader.load_model( 2025-07-20 11:25:27,753 - sglang - INFO - ^^^^^^^^^^^^^^^^^^ 2025-07-20 11:25:27,753 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^ 2025-07-20 11:25:27,753 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/model_loader/loader.py", line 362, in load_model 2025-07-20 11:25:27,753 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/model_loader/loader.py", line 362, in load_model 2025-07-20 11:25:27,753 - sglang - INFO - model.load_weights(self._get_all_weights(model_config, model)) 2025-07-20 11:25:27,753 - __main__ - INFO - model.load_weights(self._get_all_weights(model_config, model)) 2025-07-20 11:25:27,753 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/models/qwen2_vl.py", line 576, in load_weights 2025-07-20 11:25:27,753 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/models/qwen2_vl.py", line 576, in load_weights 2025-07-20 11:25:27,754 - sglang - INFO - for name, loaded_weight in weights: 2025-07-20 11:25:27,754 - __main__ - INFO - for name, loaded_weight in weights: 2025-07-20 11:25:27,754 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/model_loader/loader.py", line 335, in _get_all_weights 2025-07-20 11:25:27,754 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/model_loader/loader.py", line 335, in _get_all_weights 2025-07-20 11:25:27,754 - sglang - INFO - yield from self._get_weights_iterator(primary_weights) 2025-07-20 11:25:27,754 - __main__ - INFO - yield from self._get_weights_iterator(primary_weights) 2025-07-20 11:25:27,754 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:25:27,754 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:25:27,754 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/model_loader/loader.py", line 303, in _get_weights_iterator 2025-07-20 11:25:27,754 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/model_loader/loader.py", line 303, in _get_weights_iterator 2025-07-20 11:25:27,754 - sglang - INFO - hf_folder, hf_weights_files, use_safetensors = self._prepare_weights( 2025-07-20 11:25:27,754 - __main__ - INFO - hf_folder, hf_weights_files, use_safetensors = self._prepare_weights( 2025-07-20 11:25:27,754 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:25:27,754 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:25:27,754 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/model_loader/loader.py", line 255, in _prepare_weights 2025-07-20 11:25:27,754 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/model_loader/loader.py", line 255, in _prepare_weights 2025-07-20 11:25:27,754 - sglang - INFO - hf_folder = download_weights_from_hf( 2025-07-20 11:25:27,754 - __main__ - INFO - hf_folder = download_weights_from_hf( 2025-07-20 11:25:27,754 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:25:27,754 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:25:27,754 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/model_loader/weight_utils.py", line 246, in download_weights_from_hf 2025-07-20 11:25:27,754 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/model_loader/weight_utils.py", line 246, in download_weights_from_hf 2025-07-20 11:25:27,754 - sglang - INFO - file_list = fs.ls(model_name_or_path, detail=False, revision=revision) 2025-07-20 11:25:27,755 - __main__ - INFO - file_list = fs.ls(model_name_or_path, detail=False, revision=revision) 2025-07-20 11:25:27,755 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:25:27,755 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:25:27,755 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/huggingface_hub/hf_file_system.py", line 368, in ls 2025-07-20 11:25:27,755 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/huggingface_hub/hf_file_system.py", line 368, in ls 2025-07-20 11:25:27,755 - sglang - INFO - resolved_path = self.resolve_path(path, revision=revision) 2025-07-20 11:25:27,755 - __main__ - INFO - resolved_path = self.resolve_path(path, revision=revision) 2025-07-20 11:25:27,755 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:25:27,755 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:25:27,755 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/huggingface_hub/hf_file_system.py", line 209, in resolve_path 2025-07-20 11:25:27,755 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/huggingface_hub/hf_file_system.py", line 209, in resolve_path 2025-07-20 11:25:27,755 - sglang - INFO - repo_and_revision_exist, err = self._repo_and_revision_exist(repo_type, repo_id, revision) 2025-07-20 11:25:27,755 - __main__ - INFO - repo_and_revision_exist, err = self._repo_and_revision_exist(repo_type, repo_id, revision) 2025-07-20 11:25:27,755 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:25:27,755 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:25:27,755 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/huggingface_hub/hf_file_system.py", line 125, in _repo_and_revision_exist 2025-07-20 11:25:27,755 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/huggingface_hub/hf_file_system.py", line 125, in _repo_and_revision_exist 2025-07-20 11:25:27,755 - sglang - INFO - self._api.repo_info( 2025-07-20 11:25:27,755 - __main__ - INFO - self._api.repo_info( 2025-07-20 11:25:27,755 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/huggingface_hub/utils/_validators.py", line 114, in _inner_fn 2025-07-20 11:25:27,755 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/huggingface_hub/utils/_validators.py", line 114, in _inner_fn 2025-07-20 11:25:27,755 - sglang - INFO - return fn(*args, **kwargs) 2025-07-20 11:25:27,756 - __main__ - INFO - return fn(*args, **kwargs) 2025-07-20 11:25:27,756 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:25:27,756 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:25:27,756 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/huggingface_hub/hf_api.py", line 2816, in repo_info 2025-07-20 11:25:27,756 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/huggingface_hub/hf_api.py", line 2816, in repo_info 2025-07-20 11:25:27,756 - sglang - INFO - return method( 2025-07-20 11:25:27,756 - __main__ - INFO - return method( 2025-07-20 11:25:27,756 - sglang - INFO - ^^^^^^^ 2025-07-20 11:25:27,756 - __main__ - INFO - ^^^^^^^ 2025-07-20 11:25:27,756 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/huggingface_hub/utils/_validators.py", line 114, in _inner_fn 2025-07-20 11:25:27,756 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/huggingface_hub/utils/_validators.py", line 114, in _inner_fn 2025-07-20 11:25:27,756 - sglang - INFO - return fn(*args, **kwargs) 2025-07-20 11:25:27,756 - __main__ - INFO - return fn(*args, **kwargs) 2025-07-20 11:25:27,756 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:25:27,756 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:25:27,756 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/huggingface_hub/hf_api.py", line 2600, in model_info 2025-07-20 11:25:27,756 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/huggingface_hub/hf_api.py", line 2600, in model_info 2025-07-20 11:25:27,756 - sglang - INFO - r = get_session().get(path, headers=headers, timeout=timeout, params=params) 2025-07-20 11:25:27,756 - __main__ - INFO - r = get_session().get(path, headers=headers, timeout=timeout, params=params) 2025-07-20 11:25:27,756 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:25:27,756 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:25:27,756 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/requests/sessions.py", line 602, in get 2025-07-20 11:25:27,757 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/requests/sessions.py", line 602, in get 2025-07-20 11:25:27,757 - sglang - INFO - return self.request("GET", url, **kwargs) 2025-07-20 11:25:27,757 - __main__ - INFO - return self.request("GET", url, **kwargs) 2025-07-20 11:25:27,757 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:25:27,757 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:25:27,757 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/requests/sessions.py", line 589, in request 2025-07-20 11:25:27,757 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/requests/sessions.py", line 589, in request 2025-07-20 11:25:27,757 - sglang - INFO - resp = self.send(prep, **send_kwargs) 2025-07-20 11:25:27,757 - __main__ - INFO - resp = self.send(prep, **send_kwargs) 2025-07-20 11:25:27,757 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:25:27,757 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:25:27,757 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/requests/sessions.py", line 703, in send 2025-07-20 11:25:27,757 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/requests/sessions.py", line 703, in send 2025-07-20 11:25:27,757 - sglang - INFO - r = adapter.send(request, **kwargs) 2025-07-20 11:25:27,757 - __main__ - INFO - r = adapter.send(request, **kwargs) 2025-07-20 11:25:27,757 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:25:27,757 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:25:27,757 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/huggingface_hub/utils/_http.py", line 96, in send 2025-07-20 11:25:27,757 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/huggingface_hub/utils/_http.py", line 96, in send 2025-07-20 11:25:27,757 - sglang - INFO - return super().send(request, *args, **kwargs) 2025-07-20 11:25:27,757 - __main__ - INFO - return super().send(request, *args, **kwargs) 2025-07-20 11:25:27,757 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:25:27,757 - __main__ - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 11:25:27,758 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/requests/adapters.py", line 700, in send 2025-07-20 11:25:27,758 - __main__ - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/requests/adapters.py", line 700, in send 2025-07-20 11:25:27,758 - sglang - INFO - raise ConnectionError(e, request=request) 2025-07-20 11:25:27,758 - __main__ - INFO - raise ConnectionError(e, request=request) 2025-07-20 11:25:27,758 - sglang - INFO - requests.exceptions.ConnectionError: (MaxRetryError("HTTPSConnectionPool(host='huggingface.co', port=443): Max retries exceeded with url: /api/models/allenai/olmOCR-7B-0225-preview (Caused by NewConnectionError(': Failed to establish a new connection: [Errno 101] Network is unreachable'))"), '(Request ID: 9331f4da-f005-4022-a34a-4bb0423deb4d)') 2025-07-20 11:25:27,758 - __main__ - INFO - requests.exceptions.ConnectionError: (MaxRetryError("HTTPSConnectionPool(host='huggingface.co', port=443): Max retries exceeded with url: /api/models/allenai/olmOCR-7B-0225-preview (Caused by NewConnectionError(': Failed to establish a new connection: [Errno 101] Network is unreachable'))"), '(Request ID: 9331f4da-f005-4022-a34a-4bb0423deb4d)') 2025-07-20 11:25:27,758 - sglang - INFO - 2025-07-20 11:25:27,758 - __main__ - INFO - 2025-07-20 11:25:27,758 - sglang - INFO - [2025-07-20 11:25:27] Received sigquit from a child proces. It usually means the child failed. 2025-07-20 11:25:27,758 - __main__ - INFO - [2025-07-20 11:25:27] Received sigquit from a child proces. It usually means the child failed. 2025-07-20 11:25:28,039 - __main__ - WARNING - SGLang server task ended 2025-07-20 11:25:28,040 - __main__ - ERROR - Ended up starting the sglang server more than 5 times, cancelling pipeline 2025-07-20 11:25:28,040 - __main__ - ERROR - 2025-07-20 11:25:28,040 - __main__ - ERROR - Please make sure sglang is installed according to the latest instructions here: https://docs.sglang.ai/start/install.html 2025-07-20 15:06:33,448 - __main__ - INFO - Got --pdfs argument, going to add to the work queue 2025-07-20 15:06:33,448 - __main__ - INFO - Loading file at tests/gnarly_pdfs/horribleocr.pdf as PDF document 2025-07-20 15:06:33,448 - __main__ - INFO - Found 1 total pdf paths to add 2025-07-20 15:06:33,451 - __main__ - INFO - Calculated items_per_group: 500 based on average pages per PDF: 1.00 2025-07-20 15:06:33,678 - __main__ - INFO - Starting pipeline with PID 589922 2025-07-20 15:06:33,679 - __main__ - INFO - Using local model path at '/root/llm/olmOCR-7B-0225-preview' 2025-07-20 15:06:38,788 - __main__ - WARNING - Attempt 1: Please wait for sglang server to become ready... 2025-07-20 15:06:40,602 - sglang - INFO - [2025-07-20 15:06:40] server_args=ServerArgs(model_path='/root/llm/olmOCR-7B-0225-preview', tokenizer_path='/root/llm/olmOCR-7B-0225-preview', tokenizer_mode='auto', load_format='auto', trust_remote_code=False, dtype='auto', kv_cache_dtype='auto', quantization_param_path=None, quantization=None, context_length=None, device='cuda', served_model_name='/root/llm/olmOCR-7B-0225-preview', chat_template='qwen2-vl', is_embedding=False, revision=None, skip_tokenizer_init=False, host='127.0.0.1', port=30024, mem_fraction_static=0.8, max_running_requests=None, max_total_tokens=None, chunked_prefill_size=2048, max_prefill_tokens=16384, schedule_policy='lpm', schedule_conservativeness=1.0, cpu_offload_gb=0, prefill_only_one_req=False, tp_size=1, stream_interval=1, stream_output=False, random_seed=555501304, constrained_json_whitespace_pattern=None, watchdog_timeout=300, download_dir=None, base_gpu_id=0, log_level='info', log_level_http='warning', log_requests=False, show_time_cost=False, enable_metrics=False, decode_log_interval=40, api_key=None, file_storage_pth='sglang_storage', enable_cache_report=False, dp_size=1, load_balance_method='round_robin', ep_size=1, dist_init_addr=None, nnodes=1, node_rank=0, json_model_override_args='{}', lora_paths=None, max_loras_per_batch=8, attention_backend='flashinfer', sampling_backend='flashinfer', grammar_backend='outlines', speculative_draft_model_path=None, speculative_algorithm=None, speculative_num_steps=5, speculative_num_draft_tokens=64, speculative_eagle_topk=8, enable_double_sparsity=False, ds_channel_config_path=None, ds_heavy_channel_num=32, ds_heavy_token_num=256, ds_heavy_channel_type='qk', ds_sparse_decode_threshold=4096, disable_radix_cache=False, disable_jump_forward=False, disable_cuda_graph=False, disable_cuda_graph_padding=False, disable_outlines_disk_cache=False, disable_custom_all_reduce=False, disable_mla=False, disable_overlap_schedule=False, enable_mixed_chunk=False, enable_dp_attention=False, enable_ep_moe=False, enable_torch_compile=False, torch_compile_max_bs=32, cuda_graph_max_bs=8, cuda_graph_bs=None, torchao_config='', enable_nan_detection=False, enable_p2p_check=False, triton_attention_reduce_in_fp32=False, triton_attention_num_kv_splits=8, num_continuous_decode_steps=1, delete_ckpt_after_loading=False, enable_memory_saver=False, allow_auto_truncate=False, enable_custom_logit_processor=False, tool_call_parser=None) 2025-07-20 15:06:40,603 - __main__ - INFO - [2025-07-20 15:06:40] server_args=ServerArgs(model_path='/root/llm/olmOCR-7B-0225-preview', tokenizer_path='/root/llm/olmOCR-7B-0225-preview', tokenizer_mode='auto', load_format='auto', trust_remote_code=False, dtype='auto', kv_cache_dtype='auto', quantization_param_path=None, quantization=None, context_length=None, device='cuda', served_model_name='/root/llm/olmOCR-7B-0225-preview', chat_template='qwen2-vl', is_embedding=False, revision=None, skip_tokenizer_init=False, host='127.0.0.1', port=30024, mem_fraction_static=0.8, max_running_requests=None, max_total_tokens=None, chunked_prefill_size=2048, max_prefill_tokens=16384, schedule_policy='lpm', schedule_conservativeness=1.0, cpu_offload_gb=0, prefill_only_one_req=False, tp_size=1, stream_interval=1, stream_output=False, random_seed=555501304, constrained_json_whitespace_pattern=None, watchdog_timeout=300, download_dir=None, base_gpu_id=0, log_level='info', log_level_http='warning', log_requests=False, show_time_cost=False, enable_metrics=False, decode_log_interval=40, api_key=None, file_storage_pth='sglang_storage', enable_cache_report=False, dp_size=1, load_balance_method='round_robin', ep_size=1, dist_init_addr=None, nnodes=1, node_rank=0, json_model_override_args='{}', lora_paths=None, max_loras_per_batch=8, attention_backend='flashinfer', sampling_backend='flashinfer', grammar_backend='outlines', speculative_draft_model_path=None, speculative_algorithm=None, speculative_num_steps=5, speculative_num_draft_tokens=64, speculative_eagle_topk=8, enable_double_sparsity=False, ds_channel_config_path=None, ds_heavy_channel_num=32, ds_heavy_token_num=256, ds_heavy_channel_type='qk', ds_sparse_decode_threshold=4096, disable_radix_cache=False, disable_jump_forward=False, disable_cuda_graph=False, disable_cuda_graph_padding=False, disable_outlines_disk_cache=False, disable_custom_all_reduce=False, disable_mla=False, disable_overlap_schedule=False, enable_mixed_chunk=False, enable_dp_attention=False, enable_ep_moe=False, enable_torch_compile=False, torch_compile_max_bs=32, cuda_graph_max_bs=8, cuda_graph_bs=None, torchao_config='', enable_nan_detection=False, enable_p2p_check=False, triton_attention_reduce_in_fp32=False, triton_attention_num_kv_splits=8, num_continuous_decode_steps=1, delete_ckpt_after_loading=False, enable_memory_saver=False, allow_auto_truncate=False, enable_custom_logit_processor=False, tool_call_parser=None) 2025-07-20 15:06:41,702 - sglang - INFO - [2025-07-20 15:06:41] Use chat template for the OpenAI-compatible API server: qwen2-vl 2025-07-20 15:06:41,702 - __main__ - INFO - [2025-07-20 15:06:41] Use chat template for the OpenAI-compatible API server: qwen2-vl 2025-07-20 15:06:44,868 - __main__ - WARNING - Attempt 2: Please wait for sglang server to become ready... 2025-07-20 15:06:47,936 - sglang - INFO - [2025-07-20 15:06:47 TP0] Overlap scheduler is disabled for multimodal models. 2025-07-20 15:06:47,936 - __main__ - INFO - [2025-07-20 15:06:47 TP0] Overlap scheduler is disabled for multimodal models. 2025-07-20 15:06:47,938 - sglang - INFO - [2025-07-20 15:06:47 TP0] Automatically reduce --mem-fraction-static to 0.760 because this is a multimodal model. 2025-07-20 15:06:47,938 - __main__ - INFO - [2025-07-20 15:06:47 TP0] Automatically reduce --mem-fraction-static to 0.760 because this is a multimodal model. 2025-07-20 15:06:47,938 - sglang - INFO - [2025-07-20 15:06:47 TP0] Automatically turn off --chunked-prefill-size and disable radix cache for qwen2-vl. 2025-07-20 15:06:47,938 - __main__ - INFO - [2025-07-20 15:06:47 TP0] Automatically turn off --chunked-prefill-size and disable radix cache for qwen2-vl. 2025-07-20 15:06:47,939 - sglang - INFO - [2025-07-20 15:06:47 TP0] Init torch distributed begin. 2025-07-20 15:06:47,939 - __main__ - INFO - [2025-07-20 15:06:47 TP0] Init torch distributed begin. 2025-07-20 15:06:50,947 - __main__ - WARNING - Attempt 3: Please wait for sglang server to become ready... 2025-07-20 15:06:53,328 - sglang - INFO - [2025-07-20 15:06:53 TP0] Load weight begin. avail mem=23.33 GB 2025-07-20 15:06:53,328 - __main__ - INFO - [2025-07-20 15:06:53 TP0] Load weight begin. avail mem=23.33 GB 2025-07-20 15:06:53,875 - sglang - INFO - Loading safetensors checkpoint shards: 0% Completed | 0/4 [00:00 - Got InternalServerError from server: b'Internal Server Error', skipping this response 2025-07-20 15:47:00,135 - __main__ - INFO - Built page query for tests/gnarly_pdfs/skinnypage.pdf-2 2025-07-20 15:47:00,455 - sglang - INFO - Token indices sequence length is longer than the specified maximum sequence length for this model (78749 > 32768). Running this sequence through the model will result in indexing errors 2025-07-20 15:47:00,547 - sglang - INFO - [2025-07-20 15:47:00 TP0] Prefill batch. #new-seq: 1, #new-token: 1958, #cached-token: 0, cache hit rate: 0.00%, token usage: 0.70, #running-req: 7, #queue-req: 164 2025-07-20 15:47:00,547 - __main__ - INFO - sglang running req: 7 queue req: 164 2025-07-20 15:47:00,635 - __main__ - INFO - Finished TaskGroup for worker on 16158dc6fac58e5a41d3888b9554c3d75b2a5744 2025-07-20 15:47:00,635 - __main__ - INFO - Got 1 docs for 16158dc6fac58e5a41d3888b9554c3d75b2a5744 2025-07-20 15:47:00,638 - __main__ - INFO - Worker 4 exiting due to empty queue 2025-07-20 15:47:00,638 - __main__ - INFO - Worker 5 exiting due to empty queue 2025-07-20 15:47:00,638 - __main__ - INFO - Worker 6 exiting due to empty queue 2025-07-20 15:47:00,639 - __main__ - INFO - Worker 7 exiting due to empty queue 2025-07-20 15:47:00,639 - __main__ - INFO - Worker 2 exiting due to empty queue 2025-07-20 15:47:01,862 - sglang - INFO - [2025-07-20 15:47:01 TP0] Decode batch. #running-req: 8, #token: 28667, token usage: 0.75, gen throughput (token/s): 138.38, #queue-req: 205 2025-07-20 15:47:01,862 - __main__ - INFO - sglang running req: 8 queue req: 205 2025-07-20 15:47:03,233 - sglang - INFO - [2025-07-20 15:47:03 TP0] Decode batch. #running-req: 8, #token: 28987, token usage: 0.76, gen throughput (token/s): 233.27, #queue-req: 243 2025-07-20 15:47:03,234 - __main__ - INFO - sglang running req: 8 queue req: 243 2025-07-20 15:47:04,557 - sglang - INFO - [2025-07-20 15:47:04 TP0] Decode batch. #running-req: 8, #token: 29307, token usage: 0.77, gen throughput (token/s): 241.78, #queue-req: 282 2025-07-20 15:47:04,557 - __main__ - INFO - sglang running req: 8 queue req: 282 2025-07-20 15:47:04,744 - __main__ - INFO - Queue remaining: 0 2025-07-20 15:47:04,744 - __main__ - INFO - Metric Name Lifetime (tokens/sec) Recently (tokens/sec) ---------------------------------------------------------------------------------- finished_input_tokens 7.35 32.62 finished_output_tokens 1.76 7.83 sglang_input_tokens 904.93 895.69 sglang_output_tokens 258.39 258.44 2025-07-20 15:47:04,744 - __main__ - INFO - Worker ID | finished | started ----------+----------+-------- 0 | 497 | 500 1 | 9 | 10 2 | 5 | 5 3 | 1 | 529 2025-07-20 15:47:05,990 - sglang - INFO - [2025-07-20 15:47:05 TP0] Decode batch. #running-req: 8, #token: 29627, token usage: 0.78, gen throughput (token/s): 223.34, #queue-req: 327 2025-07-20 15:47:05,990 - __main__ - INFO - sglang running req: 8 queue req: 327 2025-07-20 15:47:07,333 - sglang - INFO - [2025-07-20 15:47:07 TP0] Decode batch. #running-req: 8, #token: 29947, token usage: 0.79, gen throughput (token/s): 238.16, #queue-req: 363 2025-07-20 15:47:07,333 - __main__ - INFO - sglang running req: 8 queue req: 363 2025-07-20 15:47:07,571 - __main__ - WARNING - JSON decode error on attempt 1 for scripts/data/11445224007035644H44421110A0001.pdf-3: Expecting ',' delimiter: line 1 column 2734 (char 2733) 2025-07-20 15:47:07,595 - sglang - INFO - [2025-07-20 15:47:07 TP0] Prefill batch. #new-seq: 1, #new-token: 3732, #cached-token: 0, cache hit rate: 0.00%, token usage: 0.66, #running-req: 7, #queue-req: 371 2025-07-20 15:47:07,595 - __main__ - INFO - sglang running req: 7 queue req: 371 2025-07-20 15:47:08,256 - __main__ - INFO - Built page query for scripts/data/11445224007035644H44421110A0001.pdf-3 2025-07-20 15:47:08,387 - sglang - INFO - [2025-07-20 15:47:08] Exception in TokenizerManager: 2025-07-20 15:47:08,388 - sglang - INFO - Traceback (most recent call last): 2025-07-20 15:47:08,388 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/managers/image_processor.py", line 417, in _process_single_image_task 2025-07-20 15:47:08,388 - sglang - INFO - process_result = image_processor(image) 2025-07-20 15:47:08,388 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 15:47:08,388 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/transformers/image_processing_utils.py", line 41, in __call__ 2025-07-20 15:47:08,388 - sglang - INFO - return self.preprocess(images, **kwargs) 2025-07-20 15:47:08,388 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 15:47:08,388 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/transformers/models/qwen2_vl/image_processing_qwen2_vl.py", line 417, in preprocess 2025-07-20 15:47:08,388 - sglang - INFO - patches, image_grid_thw = self._preprocess( 2025-07-20 15:47:08,388 - sglang - INFO - ^^^^^^^^^^^^^^^^^ 2025-07-20 15:47:08,388 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/transformers/models/qwen2_vl/image_processing_qwen2_vl.py", line 269, in _preprocess 2025-07-20 15:47:08,388 - sglang - INFO - resized_height, resized_width = smart_resize( 2025-07-20 15:47:08,389 - sglang - INFO - ^^^^^^^^^^^^^ 2025-07-20 15:47:08,389 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/transformers/models/qwen2_vl/image_processing_qwen2_vl.py", line 112, in smart_resize 2025-07-20 15:47:08,389 - sglang - INFO - raise ValueError(f"height:{height} or width:{width} must be larger than factor:{factor}") 2025-07-20 15:47:08,389 - sglang - INFO - ValueError: height:1024 or width:17 must be larger than factor:28 2025-07-20 15:47:08,389 - sglang - INFO - 2025-07-20 15:47:10,098 - sglang - INFO - [2025-07-20 15:47:10 TP0] Decode batch. #running-req: 8, #token: 28996, token usage: 0.76, gen throughput (token/s): 115.40, #queue-req: 451 2025-07-20 15:47:10,098 - __main__ - INFO - sglang running req: 8 queue req: 451 2025-07-20 15:47:10,735 - __main__ - INFO - Finished TaskGroup for worker on 8d1e4551c46000ba4529a1ac09bae565b95f4ab7 2025-07-20 15:47:10,735 - __main__ - INFO - Got 1 docs for 8d1e4551c46000ba4529a1ac09bae565b95f4ab7 2025-07-20 15:47:10,737 - __main__ - INFO - Worker 1 exiting due to empty queue 2025-07-20 15:47:10,759 - sglang - INFO - [2025-07-20 15:47:10 TP0] Prefill batch. #new-seq: 2, #new-token: 4253, #cached-token: 0, cache hit rate: 0.00%, token usage: 0.61, #running-req: 7, #queue-req: 478 2025-07-20 15:47:10,759 - __main__ - INFO - sglang running req: 7 queue req: 478 2025-07-20 15:47:11,509 - sglang - INFO - [2025-07-20 15:47:11] ERROR: Exception in ASGI application 2025-07-20 15:47:11,509 - sglang - INFO - Traceback (most recent call last): 2025-07-20 15:47:11,509 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/uvicorn/protocols/http/httptools_impl.py", line 409, in run_asgi 2025-07-20 15:47:11,509 - sglang - INFO - result = await app( # type: ignore[func-returns-value] 2025-07-20 15:47:11,510 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 15:47:11,510 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/uvicorn/middleware/proxy_headers.py", line 60, in __call__ 2025-07-20 15:47:11,510 - sglang - INFO - return await self.app(scope, receive, send) 2025-07-20 15:47:11,510 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 15:47:11,510 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/fastapi/applications.py", line 1054, in __call__ 2025-07-20 15:47:11,510 - sglang - INFO - await super().__call__(scope, receive, send) 2025-07-20 15:47:11,510 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/starlette/applications.py", line 112, in __call__ 2025-07-20 15:47:11,510 - sglang - INFO - await self.middleware_stack(scope, receive, send) 2025-07-20 15:47:11,510 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/starlette/middleware/errors.py", line 187, in __call__ 2025-07-20 15:47:11,510 - sglang - INFO - raise exc 2025-07-20 15:47:11,510 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/starlette/middleware/errors.py", line 165, in __call__ 2025-07-20 15:47:11,510 - sglang - INFO - await self.app(scope, receive, _send) 2025-07-20 15:47:11,510 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/starlette/middleware/cors.py", line 85, in __call__ 2025-07-20 15:47:11,510 - sglang - INFO - await self.app(scope, receive, send) 2025-07-20 15:47:11,511 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/starlette/middleware/exceptions.py", line 62, in __call__ 2025-07-20 15:47:11,511 - sglang - INFO - await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send) 2025-07-20 15:47:11,511 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app 2025-07-20 15:47:11,511 - sglang - INFO - raise exc 2025-07-20 15:47:11,511 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app 2025-07-20 15:47:11,511 - sglang - INFO - await app(scope, receive, sender) 2025-07-20 15:47:11,511 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/starlette/routing.py", line 714, in __call__ 2025-07-20 15:47:11,511 - sglang - INFO - await self.middleware_stack(scope, receive, send) 2025-07-20 15:47:11,511 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/starlette/routing.py", line 734, in app 2025-07-20 15:47:11,511 - sglang - INFO - await route.handle(scope, receive, send) 2025-07-20 15:47:11,511 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/starlette/routing.py", line 288, in handle 2025-07-20 15:47:11,511 - sglang - INFO - await self.app(scope, receive, send) 2025-07-20 15:47:11,511 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/starlette/routing.py", line 76, in app 2025-07-20 15:47:11,511 - sglang - INFO - await wrap_app_handling_exceptions(app, request)(scope, receive, send) 2025-07-20 15:47:11,511 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app 2025-07-20 15:47:11,512 - sglang - INFO - raise exc 2025-07-20 15:47:11,512 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app 2025-07-20 15:47:11,512 - sglang - INFO - await app(scope, receive, sender) 2025-07-20 15:47:11,512 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/starlette/routing.py", line 73, in app 2025-07-20 15:47:11,512 - sglang - INFO - response = await f(request) 2025-07-20 15:47:11,512 - sglang - INFO - ^^^^^^^^^^^^^^^^ 2025-07-20 15:47:11,512 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/fastapi/routing.py", line 301, in app 2025-07-20 15:47:11,512 - sglang - INFO - raw_response = await run_endpoint_function( 2025-07-20 15:47:11,512 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 15:47:11,512 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/fastapi/routing.py", line 212, in run_endpoint_function 2025-07-20 15:47:11,512 - sglang - INFO - return await dependant.call(**values) 2025-07-20 15:47:11,512 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 15:47:11,512 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/entrypoints/http_server.py", line 406, in openai_v1_chat_completions 2025-07-20 15:47:11,512 - sglang - INFO - return await v1_chat_completions(_global_state.tokenizer_manager, raw_request) 2025-07-20 15:47:11,513 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 15:47:11,513 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/openai_api/adapter.py", line 1426, in v1_chat_completions 2025-07-20 15:47:11,513 - sglang - INFO - ret = await tokenizer_manager.generate_request( 2025-07-20 15:47:11,513 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 15:47:11,513 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/managers/tokenizer_manager.py", line 291, in generate_request 2025-07-20 15:47:11,513 - sglang - INFO - tokenized_obj = await self._tokenize_one_request(obj) 2025-07-20 15:47:11,513 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 15:47:11,513 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/managers/tokenizer_manager.py", line 331, in _tokenize_one_request 2025-07-20 15:47:11,513 - sglang - INFO - image_inputs: Dict = await self.image_processor.process_images_async( 2025-07-20 15:47:11,513 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 15:47:11,513 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/managers/image_processor.py", line 474, in process_images_async 2025-07-20 15:47:11,513 - sglang - INFO - pixel_values, image_hash, image_size, image_grid_thw = ( 2025-07-20 15:47:11,513 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 15:47:11,513 - sglang - INFO - TypeError: cannot unpack non-iterable NoneType object 2025-07-20 15:47:11,514 - __main__ - WARNING - ValueError on attempt 1 for tests/gnarly_pdfs/skinnypage.pdf-2: - Got InternalServerError from server: b'Internal Server Error', skipping this response 2025-07-20 15:47:11,653 - __main__ - WARNING - ValueError on attempt 0 for tests/gnarly_pdfs/map1.pdf-1: - Got BadRequestError from server: b'{"object":"error","message":"The input (78749 tokens) is longer than the model\'s context length (32768 tokens).","type":"BadRequestError","param":null,"code":400}', skipping this response 2025-07-20 15:47:11,738 - __main__ - INFO - Built page query for tests/gnarly_pdfs/skinnypage.pdf-2 2025-07-20 15:47:11,767 - sglang - INFO - [2025-07-20 15:47:11] Exception in TokenizerManager: 2025-07-20 15:47:11,767 - sglang - INFO - Traceback (most recent call last): 2025-07-20 15:47:11,767 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/managers/image_processor.py", line 417, in _process_single_image_task 2025-07-20 15:47:11,767 - sglang - INFO - process_result = image_processor(image) 2025-07-20 15:47:11,767 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 15:47:11,767 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/transformers/image_processing_utils.py", line 41, in __call__ 2025-07-20 15:47:11,768 - sglang - INFO - return self.preprocess(images, **kwargs) 2025-07-20 15:47:11,768 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 15:47:11,768 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/transformers/models/qwen2_vl/image_processing_qwen2_vl.py", line 417, in preprocess 2025-07-20 15:47:11,768 - sglang - INFO - patches, image_grid_thw = self._preprocess( 2025-07-20 15:47:11,768 - sglang - INFO - ^^^^^^^^^^^^^^^^^ 2025-07-20 15:47:11,768 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/transformers/models/qwen2_vl/image_processing_qwen2_vl.py", line 269, in _preprocess 2025-07-20 15:47:11,768 - sglang - INFO - resized_height, resized_width = smart_resize( 2025-07-20 15:47:11,768 - sglang - INFO - ^^^^^^^^^^^^^ 2025-07-20 15:47:11,768 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/transformers/models/qwen2_vl/image_processing_qwen2_vl.py", line 112, in smart_resize 2025-07-20 15:47:11,768 - sglang - INFO - raise ValueError(f"height:{height} or width:{width} must be larger than factor:{factor}") 2025-07-20 15:47:11,768 - sglang - INFO - ValueError: height:1024 or width:17 must be larger than factor:28 2025-07-20 15:47:11,768 - sglang - INFO - 2025-07-20 15:47:11,770 - sglang - INFO - [2025-07-20 15:47:11] ERROR: Exception in ASGI application 2025-07-20 15:47:11,770 - sglang - INFO - Traceback (most recent call last): 2025-07-20 15:47:11,770 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/uvicorn/protocols/http/httptools_impl.py", line 409, in run_asgi 2025-07-20 15:47:11,770 - sglang - INFO - result = await app( # type: ignore[func-returns-value] 2025-07-20 15:47:11,770 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 15:47:11,770 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/uvicorn/middleware/proxy_headers.py", line 60, in __call__ 2025-07-20 15:47:11,770 - sglang - INFO - return await self.app(scope, receive, send) 2025-07-20 15:47:11,771 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 15:47:11,771 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/fastapi/applications.py", line 1054, in __call__ 2025-07-20 15:47:11,771 - sglang - INFO - await super().__call__(scope, receive, send) 2025-07-20 15:47:11,771 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/starlette/applications.py", line 112, in __call__ 2025-07-20 15:47:11,771 - sglang - INFO - await self.middleware_stack(scope, receive, send) 2025-07-20 15:47:11,771 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/starlette/middleware/errors.py", line 187, in __call__ 2025-07-20 15:47:11,771 - sglang - INFO - raise exc 2025-07-20 15:47:11,771 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/starlette/middleware/errors.py", line 165, in __call__ 2025-07-20 15:47:11,771 - sglang - INFO - await self.app(scope, receive, _send) 2025-07-20 15:47:11,771 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/starlette/middleware/cors.py", line 85, in __call__ 2025-07-20 15:47:11,771 - sglang - INFO - await self.app(scope, receive, send) 2025-07-20 15:47:11,771 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/starlette/middleware/exceptions.py", line 62, in __call__ 2025-07-20 15:47:11,771 - sglang - INFO - await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send) 2025-07-20 15:47:11,771 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app 2025-07-20 15:47:11,771 - sglang - INFO - raise exc 2025-07-20 15:47:11,772 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app 2025-07-20 15:47:11,772 - sglang - INFO - await app(scope, receive, sender) 2025-07-20 15:47:11,772 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/starlette/routing.py", line 714, in __call__ 2025-07-20 15:47:11,772 - sglang - INFO - await self.middleware_stack(scope, receive, send) 2025-07-20 15:47:11,772 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/starlette/routing.py", line 734, in app 2025-07-20 15:47:11,772 - sglang - INFO - await route.handle(scope, receive, send) 2025-07-20 15:47:11,772 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/starlette/routing.py", line 288, in handle 2025-07-20 15:47:11,772 - sglang - INFO - await self.app(scope, receive, send) 2025-07-20 15:47:11,772 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/starlette/routing.py", line 76, in app 2025-07-20 15:47:11,772 - sglang - INFO - await wrap_app_handling_exceptions(app, request)(scope, receive, send) 2025-07-20 15:47:11,772 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app 2025-07-20 15:47:11,772 - sglang - INFO - raise exc 2025-07-20 15:47:11,772 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app 2025-07-20 15:47:11,772 - sglang - INFO - await app(scope, receive, sender) 2025-07-20 15:47:11,773 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/starlette/routing.py", line 73, in app 2025-07-20 15:47:11,773 - sglang - INFO - response = await f(request) 2025-07-20 15:47:11,773 - sglang - INFO - ^^^^^^^^^^^^^^^^ 2025-07-20 15:47:11,773 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/fastapi/routing.py", line 301, in app 2025-07-20 15:47:11,773 - sglang - INFO - raw_response = await run_endpoint_function( 2025-07-20 15:47:11,773 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 15:47:11,773 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/fastapi/routing.py", line 212, in run_endpoint_function 2025-07-20 15:47:11,773 - sglang - INFO - return await dependant.call(**values) 2025-07-20 15:47:11,773 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 15:47:11,773 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/entrypoints/http_server.py", line 406, in openai_v1_chat_completions 2025-07-20 15:47:11,773 - sglang - INFO - return await v1_chat_completions(_global_state.tokenizer_manager, raw_request) 2025-07-20 15:47:11,773 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 15:47:11,773 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/openai_api/adapter.py", line 1426, in v1_chat_completions 2025-07-20 15:47:11,773 - sglang - INFO - ret = await tokenizer_manager.generate_request( 2025-07-20 15:47:11,773 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 15:47:11,774 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/managers/tokenizer_manager.py", line 291, in generate_request 2025-07-20 15:47:11,774 - sglang - INFO - tokenized_obj = await self._tokenize_one_request(obj) 2025-07-20 15:47:11,774 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 15:47:11,774 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/managers/tokenizer_manager.py", line 331, in _tokenize_one_request 2025-07-20 15:47:11,774 - sglang - INFO - image_inputs: Dict = await self.image_processor.process_images_async( 2025-07-20 15:47:11,774 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 15:47:11,774 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/managers/image_processor.py", line 474, in process_images_async 2025-07-20 15:47:11,774 - sglang - INFO - pixel_values, image_hash, image_size, image_grid_thw = ( 2025-07-20 15:47:11,774 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 15:47:11,774 - sglang - INFO - TypeError: cannot unpack non-iterable NoneType object 2025-07-20 15:47:11,775 - __main__ - WARNING - ValueError on attempt 2 for tests/gnarly_pdfs/skinnypage.pdf-2: - Got InternalServerError from server: b'Internal Server Error', skipping this response 2025-07-20 15:47:12,085 - __main__ - INFO - Built page query for tests/gnarly_pdfs/skinnypage.pdf-2 2025-07-20 15:47:12,117 - sglang - INFO - [2025-07-20 15:47:12] Exception in TokenizerManager: 2025-07-20 15:47:12,117 - sglang - INFO - Traceback (most recent call last): 2025-07-20 15:47:12,117 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/managers/image_processor.py", line 417, in _process_single_image_task 2025-07-20 15:47:12,117 - sglang - INFO - process_result = image_processor(image) 2025-07-20 15:47:12,117 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 15:47:12,117 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/transformers/image_processing_utils.py", line 41, in __call__ 2025-07-20 15:47:12,117 - sglang - INFO - return self.preprocess(images, **kwargs) 2025-07-20 15:47:12,117 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 15:47:12,117 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/transformers/models/qwen2_vl/image_processing_qwen2_vl.py", line 417, in preprocess 2025-07-20 15:47:12,117 - sglang - INFO - patches, image_grid_thw = self._preprocess( 2025-07-20 15:47:12,117 - sglang - INFO - ^^^^^^^^^^^^^^^^^ 2025-07-20 15:47:12,118 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/transformers/models/qwen2_vl/image_processing_qwen2_vl.py", line 269, in _preprocess 2025-07-20 15:47:12,118 - sglang - INFO - resized_height, resized_width = smart_resize( 2025-07-20 15:47:12,118 - sglang - INFO - ^^^^^^^^^^^^^ 2025-07-20 15:47:12,118 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/transformers/models/qwen2_vl/image_processing_qwen2_vl.py", line 112, in smart_resize 2025-07-20 15:47:12,118 - sglang - INFO - raise ValueError(f"height:{height} or width:{width} must be larger than factor:{factor}") 2025-07-20 15:47:12,118 - sglang - INFO - ValueError: height:1024 or width:17 must be larger than factor:28 2025-07-20 15:47:12,118 - sglang - INFO - 2025-07-20 15:47:12,120 - sglang - INFO - [2025-07-20 15:47:12] ERROR: Exception in ASGI application 2025-07-20 15:47:12,120 - sglang - INFO - Traceback (most recent call last): 2025-07-20 15:47:12,120 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/uvicorn/protocols/http/httptools_impl.py", line 409, in run_asgi 2025-07-20 15:47:12,120 - sglang - INFO - result = await app( # type: ignore[func-returns-value] 2025-07-20 15:47:12,120 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 15:47:12,120 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/uvicorn/middleware/proxy_headers.py", line 60, in __call__ 2025-07-20 15:47:12,120 - sglang - INFO - return await self.app(scope, receive, send) 2025-07-20 15:47:12,120 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 15:47:12,120 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/fastapi/applications.py", line 1054, in __call__ 2025-07-20 15:47:12,120 - sglang - INFO - await super().__call__(scope, receive, send) 2025-07-20 15:47:12,120 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/starlette/applications.py", line 112, in __call__ 2025-07-20 15:47:12,121 - sglang - INFO - await self.middleware_stack(scope, receive, send) 2025-07-20 15:47:12,121 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/starlette/middleware/errors.py", line 187, in __call__ 2025-07-20 15:47:12,121 - sglang - INFO - raise exc 2025-07-20 15:47:12,121 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/starlette/middleware/errors.py", line 165, in __call__ 2025-07-20 15:47:12,121 - sglang - INFO - await self.app(scope, receive, _send) 2025-07-20 15:47:12,121 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/starlette/middleware/cors.py", line 85, in __call__ 2025-07-20 15:47:12,121 - sglang - INFO - await self.app(scope, receive, send) 2025-07-20 15:47:12,121 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/starlette/middleware/exceptions.py", line 62, in __call__ 2025-07-20 15:47:12,121 - sglang - INFO - await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send) 2025-07-20 15:47:12,121 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app 2025-07-20 15:47:12,121 - sglang - INFO - raise exc 2025-07-20 15:47:12,121 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app 2025-07-20 15:47:12,121 - sglang - INFO - await app(scope, receive, sender) 2025-07-20 15:47:12,121 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/starlette/routing.py", line 714, in __call__ 2025-07-20 15:47:12,122 - sglang - INFO - await self.middleware_stack(scope, receive, send) 2025-07-20 15:47:12,122 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/starlette/routing.py", line 734, in app 2025-07-20 15:47:12,122 - sglang - INFO - await route.handle(scope, receive, send) 2025-07-20 15:47:12,122 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/starlette/routing.py", line 288, in handle 2025-07-20 15:47:12,122 - sglang - INFO - await self.app(scope, receive, send) 2025-07-20 15:47:12,122 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/starlette/routing.py", line 76, in app 2025-07-20 15:47:12,122 - sglang - INFO - await wrap_app_handling_exceptions(app, request)(scope, receive, send) 2025-07-20 15:47:12,122 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app 2025-07-20 15:47:12,122 - sglang - INFO - raise exc 2025-07-20 15:47:12,122 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app 2025-07-20 15:47:12,122 - sglang - INFO - await app(scope, receive, sender) 2025-07-20 15:47:12,122 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/starlette/routing.py", line 73, in app 2025-07-20 15:47:12,122 - sglang - INFO - response = await f(request) 2025-07-20 15:47:12,122 - sglang - INFO - ^^^^^^^^^^^^^^^^ 2025-07-20 15:47:12,122 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/fastapi/routing.py", line 301, in app 2025-07-20 15:47:12,122 - sglang - INFO - raw_response = await run_endpoint_function( 2025-07-20 15:47:12,123 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 15:47:12,123 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/fastapi/routing.py", line 212, in run_endpoint_function 2025-07-20 15:47:12,123 - sglang - INFO - return await dependant.call(**values) 2025-07-20 15:47:12,123 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 15:47:12,123 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/entrypoints/http_server.py", line 406, in openai_v1_chat_completions 2025-07-20 15:47:12,123 - sglang - INFO - return await v1_chat_completions(_global_state.tokenizer_manager, raw_request) 2025-07-20 15:47:12,123 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 15:47:12,123 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/openai_api/adapter.py", line 1426, in v1_chat_completions 2025-07-20 15:47:12,123 - sglang - INFO - ret = await tokenizer_manager.generate_request( 2025-07-20 15:47:12,123 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 15:47:12,123 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/managers/tokenizer_manager.py", line 291, in generate_request 2025-07-20 15:47:12,123 - sglang - INFO - tokenized_obj = await self._tokenize_one_request(obj) 2025-07-20 15:47:12,123 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 15:47:12,123 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/managers/tokenizer_manager.py", line 331, in _tokenize_one_request 2025-07-20 15:47:12,123 - sglang - INFO - image_inputs: Dict = await self.image_processor.process_images_async( 2025-07-20 15:47:12,124 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 15:47:12,124 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/managers/image_processor.py", line 474, in process_images_async 2025-07-20 15:47:12,124 - sglang - INFO - pixel_values, image_hash, image_size, image_grid_thw = ( 2025-07-20 15:47:12,124 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 15:47:12,124 - sglang - INFO - TypeError: cannot unpack non-iterable NoneType object 2025-07-20 15:47:12,132 - __main__ - WARNING - ValueError on attempt 3 for tests/gnarly_pdfs/skinnypage.pdf-2: - Got InternalServerError from server: b'Internal Server Error', skipping this response 2025-07-20 15:47:12,592 - __main__ - INFO - Built page query for tests/gnarly_pdfs/skinnypage.pdf-2 2025-07-20 15:47:12,620 - sglang - INFO - [2025-07-20 15:47:12] Exception in TokenizerManager: 2025-07-20 15:47:12,620 - sglang - INFO - Traceback (most recent call last): 2025-07-20 15:47:12,621 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/managers/image_processor.py", line 417, in _process_single_image_task 2025-07-20 15:47:12,621 - sglang - INFO - process_result = image_processor(image) 2025-07-20 15:47:12,621 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 15:47:12,621 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/transformers/image_processing_utils.py", line 41, in __call__ 2025-07-20 15:47:12,621 - sglang - INFO - return self.preprocess(images, **kwargs) 2025-07-20 15:47:12,621 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 15:47:12,621 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/transformers/models/qwen2_vl/image_processing_qwen2_vl.py", line 417, in preprocess 2025-07-20 15:47:12,621 - sglang - INFO - patches, image_grid_thw = self._preprocess( 2025-07-20 15:47:12,621 - sglang - INFO - ^^^^^^^^^^^^^^^^^ 2025-07-20 15:47:12,621 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/transformers/models/qwen2_vl/image_processing_qwen2_vl.py", line 269, in _preprocess 2025-07-20 15:47:12,621 - sglang - INFO - resized_height, resized_width = smart_resize( 2025-07-20 15:47:12,621 - sglang - INFO - ^^^^^^^^^^^^^ 2025-07-20 15:47:12,621 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/transformers/models/qwen2_vl/image_processing_qwen2_vl.py", line 112, in smart_resize 2025-07-20 15:47:12,621 - sglang - INFO - raise ValueError(f"height:{height} or width:{width} must be larger than factor:{factor}") 2025-07-20 15:47:12,622 - sglang - INFO - ValueError: height:1024 or width:17 must be larger than factor:28 2025-07-20 15:47:12,622 - sglang - INFO - 2025-07-20 15:47:12,634 - sglang - INFO - [2025-07-20 15:47:12] ERROR: Exception in ASGI application 2025-07-20 15:47:12,634 - sglang - INFO - Traceback (most recent call last): 2025-07-20 15:47:12,634 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/uvicorn/protocols/http/httptools_impl.py", line 409, in run_asgi 2025-07-20 15:47:12,634 - sglang - INFO - result = await app( # type: ignore[func-returns-value] 2025-07-20 15:47:12,634 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 15:47:12,634 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/uvicorn/middleware/proxy_headers.py", line 60, in __call__ 2025-07-20 15:47:12,634 - sglang - INFO - return await self.app(scope, receive, send) 2025-07-20 15:47:12,635 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 15:47:12,635 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/fastapi/applications.py", line 1054, in __call__ 2025-07-20 15:47:12,635 - sglang - INFO - await super().__call__(scope, receive, send) 2025-07-20 15:47:12,635 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/starlette/applications.py", line 112, in __call__ 2025-07-20 15:47:12,635 - sglang - INFO - await self.middleware_stack(scope, receive, send) 2025-07-20 15:47:12,635 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/starlette/middleware/errors.py", line 187, in __call__ 2025-07-20 15:47:12,635 - sglang - INFO - raise exc 2025-07-20 15:47:12,635 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/starlette/middleware/errors.py", line 165, in __call__ 2025-07-20 15:47:12,635 - sglang - INFO - await self.app(scope, receive, _send) 2025-07-20 15:47:12,635 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/starlette/middleware/cors.py", line 85, in __call__ 2025-07-20 15:47:12,635 - sglang - INFO - await self.app(scope, receive, send) 2025-07-20 15:47:12,635 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/starlette/middleware/exceptions.py", line 62, in __call__ 2025-07-20 15:47:12,635 - sglang - INFO - await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send) 2025-07-20 15:47:12,635 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app 2025-07-20 15:47:12,635 - sglang - INFO - raise exc 2025-07-20 15:47:12,636 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app 2025-07-20 15:47:12,636 - sglang - INFO - await app(scope, receive, sender) 2025-07-20 15:47:12,636 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/starlette/routing.py", line 714, in __call__ 2025-07-20 15:47:12,636 - sglang - INFO - await self.middleware_stack(scope, receive, send) 2025-07-20 15:47:12,636 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/starlette/routing.py", line 734, in app 2025-07-20 15:47:12,636 - sglang - INFO - await route.handle(scope, receive, send) 2025-07-20 15:47:12,636 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/starlette/routing.py", line 288, in handle 2025-07-20 15:47:12,636 - sglang - INFO - await self.app(scope, receive, send) 2025-07-20 15:47:12,636 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/starlette/routing.py", line 76, in app 2025-07-20 15:47:12,636 - sglang - INFO - await wrap_app_handling_exceptions(app, request)(scope, receive, send) 2025-07-20 15:47:12,636 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app 2025-07-20 15:47:12,636 - sglang - INFO - raise exc 2025-07-20 15:47:12,636 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app 2025-07-20 15:47:12,636 - sglang - INFO - await app(scope, receive, sender) 2025-07-20 15:47:12,637 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/starlette/routing.py", line 73, in app 2025-07-20 15:47:12,637 - sglang - INFO - response = await f(request) 2025-07-20 15:47:12,637 - sglang - INFO - ^^^^^^^^^^^^^^^^ 2025-07-20 15:47:12,637 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/fastapi/routing.py", line 301, in app 2025-07-20 15:47:12,637 - sglang - INFO - raw_response = await run_endpoint_function( 2025-07-20 15:47:12,637 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 15:47:12,637 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/fastapi/routing.py", line 212, in run_endpoint_function 2025-07-20 15:47:12,637 - sglang - INFO - return await dependant.call(**values) 2025-07-20 15:47:12,637 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 15:47:12,637 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/entrypoints/http_server.py", line 406, in openai_v1_chat_completions 2025-07-20 15:47:12,637 - sglang - INFO - return await v1_chat_completions(_global_state.tokenizer_manager, raw_request) 2025-07-20 15:47:12,637 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 15:47:12,637 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/openai_api/adapter.py", line 1426, in v1_chat_completions 2025-07-20 15:47:12,637 - sglang - INFO - ret = await tokenizer_manager.generate_request( 2025-07-20 15:47:12,637 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 15:47:12,638 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/managers/tokenizer_manager.py", line 291, in generate_request 2025-07-20 15:47:12,638 - sglang - INFO - tokenized_obj = await self._tokenize_one_request(obj) 2025-07-20 15:47:12,638 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 15:47:12,638 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/managers/tokenizer_manager.py", line 331, in _tokenize_one_request 2025-07-20 15:47:12,638 - sglang - INFO - image_inputs: Dict = await self.image_processor.process_images_async( 2025-07-20 15:47:12,638 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 15:47:12,638 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/managers/image_processor.py", line 474, in process_images_async 2025-07-20 15:47:12,638 - sglang - INFO - pixel_values, image_hash, image_size, image_grid_thw = ( 2025-07-20 15:47:12,638 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 15:47:12,638 - sglang - INFO - TypeError: cannot unpack non-iterable NoneType object 2025-07-20 15:47:12,638 - __main__ - WARNING - ValueError on attempt 4 for tests/gnarly_pdfs/skinnypage.pdf-2: - Got InternalServerError from server: b'Internal Server Error', skipping this response 2025-07-20 15:47:12,936 - sglang - INFO - [2025-07-20 15:47:12 TP0] Decode batch. #running-req: 9, #token: 27603, token usage: 0.73, gen throughput (token/s): 119.10, #queue-req: 520 2025-07-20 15:47:12,936 - __main__ - INFO - sglang running req: 9 queue req: 520 2025-07-20 15:47:12,965 - __main__ - INFO - Built page query for tests/gnarly_pdfs/skinnypage.pdf-2 2025-07-20 15:47:12,999 - sglang - INFO - [2025-07-20 15:47:12] Exception in TokenizerManager: 2025-07-20 15:47:12,999 - sglang - INFO - Traceback (most recent call last): 2025-07-20 15:47:12,999 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/managers/image_processor.py", line 417, in _process_single_image_task 2025-07-20 15:47:12,999 - sglang - INFO - process_result = image_processor(image) 2025-07-20 15:47:12,999 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 15:47:12,999 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/transformers/image_processing_utils.py", line 41, in __call__ 2025-07-20 15:47:12,999 - sglang - INFO - return self.preprocess(images, **kwargs) 2025-07-20 15:47:12,999 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 15:47:12,999 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/transformers/models/qwen2_vl/image_processing_qwen2_vl.py", line 417, in preprocess 2025-07-20 15:47:13,000 - sglang - INFO - patches, image_grid_thw = self._preprocess( 2025-07-20 15:47:13,000 - sglang - INFO - ^^^^^^^^^^^^^^^^^ 2025-07-20 15:47:13,000 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/transformers/models/qwen2_vl/image_processing_qwen2_vl.py", line 269, in _preprocess 2025-07-20 15:47:13,000 - sglang - INFO - resized_height, resized_width = smart_resize( 2025-07-20 15:47:13,000 - sglang - INFO - ^^^^^^^^^^^^^ 2025-07-20 15:47:13,000 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/transformers/models/qwen2_vl/image_processing_qwen2_vl.py", line 112, in smart_resize 2025-07-20 15:47:13,000 - sglang - INFO - raise ValueError(f"height:{height} or width:{width} must be larger than factor:{factor}") 2025-07-20 15:47:13,000 - sglang - INFO - ValueError: height:1024 or width:17 must be larger than factor:28 2025-07-20 15:47:13,000 - sglang - INFO - 2025-07-20 15:47:13,004 - sglang - INFO - [2025-07-20 15:47:13] ERROR: Exception in ASGI application 2025-07-20 15:47:13,004 - sglang - INFO - Traceback (most recent call last): 2025-07-20 15:47:13,004 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/uvicorn/protocols/http/httptools_impl.py", line 409, in run_asgi 2025-07-20 15:47:13,004 - sglang - INFO - result = await app( # type: ignore[func-returns-value] 2025-07-20 15:47:13,004 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 15:47:13,004 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/uvicorn/middleware/proxy_headers.py", line 60, in __call__ 2025-07-20 15:47:13,005 - sglang - INFO - return await self.app(scope, receive, send) 2025-07-20 15:47:13,005 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 15:47:13,005 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/fastapi/applications.py", line 1054, in __call__ 2025-07-20 15:47:13,005 - sglang - INFO - await super().__call__(scope, receive, send) 2025-07-20 15:47:13,005 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/starlette/applications.py", line 112, in __call__ 2025-07-20 15:47:13,005 - sglang - INFO - await self.middleware_stack(scope, receive, send) 2025-07-20 15:47:13,005 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/starlette/middleware/errors.py", line 187, in __call__ 2025-07-20 15:47:13,005 - sglang - INFO - raise exc 2025-07-20 15:47:13,005 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/starlette/middleware/errors.py", line 165, in __call__ 2025-07-20 15:47:13,005 - sglang - INFO - await self.app(scope, receive, _send) 2025-07-20 15:47:13,005 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/starlette/middleware/cors.py", line 85, in __call__ 2025-07-20 15:47:13,005 - sglang - INFO - await self.app(scope, receive, send) 2025-07-20 15:47:13,005 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/starlette/middleware/exceptions.py", line 62, in __call__ 2025-07-20 15:47:13,005 - sglang - INFO - await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send) 2025-07-20 15:47:13,005 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app 2025-07-20 15:47:13,006 - sglang - INFO - raise exc 2025-07-20 15:47:13,006 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app 2025-07-20 15:47:13,006 - sglang - INFO - await app(scope, receive, sender) 2025-07-20 15:47:13,006 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/starlette/routing.py", line 714, in __call__ 2025-07-20 15:47:13,006 - sglang - INFO - await self.middleware_stack(scope, receive, send) 2025-07-20 15:47:13,006 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/starlette/routing.py", line 734, in app 2025-07-20 15:47:13,006 - sglang - INFO - await route.handle(scope, receive, send) 2025-07-20 15:47:13,006 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/starlette/routing.py", line 288, in handle 2025-07-20 15:47:13,006 - sglang - INFO - await self.app(scope, receive, send) 2025-07-20 15:47:13,006 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/starlette/routing.py", line 76, in app 2025-07-20 15:47:13,006 - sglang - INFO - await wrap_app_handling_exceptions(app, request)(scope, receive, send) 2025-07-20 15:47:13,006 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app 2025-07-20 15:47:13,006 - sglang - INFO - raise exc 2025-07-20 15:47:13,006 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app 2025-07-20 15:47:13,007 - sglang - INFO - await app(scope, receive, sender) 2025-07-20 15:47:13,007 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/starlette/routing.py", line 73, in app 2025-07-20 15:47:13,007 - sglang - INFO - response = await f(request) 2025-07-20 15:47:13,007 - sglang - INFO - ^^^^^^^^^^^^^^^^ 2025-07-20 15:47:13,007 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/fastapi/routing.py", line 301, in app 2025-07-20 15:47:13,007 - sglang - INFO - raw_response = await run_endpoint_function( 2025-07-20 15:47:13,007 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 15:47:13,007 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/fastapi/routing.py", line 212, in run_endpoint_function 2025-07-20 15:47:13,007 - sglang - INFO - return await dependant.call(**values) 2025-07-20 15:47:13,007 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 15:47:13,007 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/entrypoints/http_server.py", line 406, in openai_v1_chat_completions 2025-07-20 15:47:13,007 - sglang - INFO - return await v1_chat_completions(_global_state.tokenizer_manager, raw_request) 2025-07-20 15:47:13,007 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 15:47:13,007 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/openai_api/adapter.py", line 1426, in v1_chat_completions 2025-07-20 15:47:13,007 - sglang - INFO - ret = await tokenizer_manager.generate_request( 2025-07-20 15:47:13,008 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 15:47:13,008 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/managers/tokenizer_manager.py", line 291, in generate_request 2025-07-20 15:47:13,008 - sglang - INFO - tokenized_obj = await self._tokenize_one_request(obj) 2025-07-20 15:47:13,008 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 15:47:13,008 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/managers/tokenizer_manager.py", line 331, in _tokenize_one_request 2025-07-20 15:47:13,008 - sglang - INFO - image_inputs: Dict = await self.image_processor.process_images_async( 2025-07-20 15:47:13,008 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 15:47:13,008 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/managers/image_processor.py", line 474, in process_images_async 2025-07-20 15:47:13,008 - sglang - INFO - pixel_values, image_hash, image_size, image_grid_thw = ( 2025-07-20 15:47:13,008 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 15:47:13,008 - sglang - INFO - TypeError: cannot unpack non-iterable NoneType object 2025-07-20 15:47:13,009 - __main__ - WARNING - ValueError on attempt 5 for tests/gnarly_pdfs/skinnypage.pdf-2: - Got InternalServerError from server: b'Internal Server Error', skipping this response 2025-07-20 15:47:13,362 - __main__ - INFO - Built page query for tests/gnarly_pdfs/skinnypage.pdf-2 2025-07-20 15:47:13,390 - sglang - INFO - [2025-07-20 15:47:13] Exception in TokenizerManager: 2025-07-20 15:47:13,390 - sglang - INFO - Traceback (most recent call last): 2025-07-20 15:47:13,390 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/managers/image_processor.py", line 417, in _process_single_image_task 2025-07-20 15:47:13,390 - sglang - INFO - process_result = image_processor(image) 2025-07-20 15:47:13,390 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 15:47:13,390 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/transformers/image_processing_utils.py", line 41, in __call__ 2025-07-20 15:47:13,390 - sglang - INFO - return self.preprocess(images, **kwargs) 2025-07-20 15:47:13,390 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 15:47:13,390 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/transformers/models/qwen2_vl/image_processing_qwen2_vl.py", line 417, in preprocess 2025-07-20 15:47:13,390 - sglang - INFO - patches, image_grid_thw = self._preprocess( 2025-07-20 15:47:13,390 - sglang - INFO - ^^^^^^^^^^^^^^^^^ 2025-07-20 15:47:13,391 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/transformers/models/qwen2_vl/image_processing_qwen2_vl.py", line 269, in _preprocess 2025-07-20 15:47:13,391 - sglang - INFO - resized_height, resized_width = smart_resize( 2025-07-20 15:47:13,391 - sglang - INFO - ^^^^^^^^^^^^^ 2025-07-20 15:47:13,391 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/transformers/models/qwen2_vl/image_processing_qwen2_vl.py", line 112, in smart_resize 2025-07-20 15:47:13,391 - sglang - INFO - raise ValueError(f"height:{height} or width:{width} must be larger than factor:{factor}") 2025-07-20 15:47:13,391 - sglang - INFO - ValueError: height:1024 or width:17 must be larger than factor:28 2025-07-20 15:47:13,391 - sglang - INFO - 2025-07-20 15:47:13,394 - sglang - INFO - [2025-07-20 15:47:13] ERROR: Exception in ASGI application 2025-07-20 15:47:13,395 - sglang - INFO - Traceback (most recent call last): 2025-07-20 15:47:13,395 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/uvicorn/protocols/http/httptools_impl.py", line 409, in run_asgi 2025-07-20 15:47:13,395 - sglang - INFO - result = await app( # type: ignore[func-returns-value] 2025-07-20 15:47:13,395 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 15:47:13,395 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/uvicorn/middleware/proxy_headers.py", line 60, in __call__ 2025-07-20 15:47:13,395 - sglang - INFO - return await self.app(scope, receive, send) 2025-07-20 15:47:13,395 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 15:47:13,395 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/fastapi/applications.py", line 1054, in __call__ 2025-07-20 15:47:13,395 - sglang - INFO - await super().__call__(scope, receive, send) 2025-07-20 15:47:13,395 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/starlette/applications.py", line 112, in __call__ 2025-07-20 15:47:13,395 - sglang - INFO - await self.middleware_stack(scope, receive, send) 2025-07-20 15:47:13,395 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/starlette/middleware/errors.py", line 187, in __call__ 2025-07-20 15:47:13,395 - sglang - INFO - raise exc 2025-07-20 15:47:13,395 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/starlette/middleware/errors.py", line 165, in __call__ 2025-07-20 15:47:13,396 - sglang - INFO - await self.app(scope, receive, _send) 2025-07-20 15:47:13,396 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/starlette/middleware/cors.py", line 85, in __call__ 2025-07-20 15:47:13,396 - sglang - INFO - await self.app(scope, receive, send) 2025-07-20 15:47:13,396 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/starlette/middleware/exceptions.py", line 62, in __call__ 2025-07-20 15:47:13,396 - sglang - INFO - await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send) 2025-07-20 15:47:13,396 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app 2025-07-20 15:47:13,396 - sglang - INFO - raise exc 2025-07-20 15:47:13,396 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app 2025-07-20 15:47:13,396 - sglang - INFO - await app(scope, receive, sender) 2025-07-20 15:47:13,396 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/starlette/routing.py", line 714, in __call__ 2025-07-20 15:47:13,396 - sglang - INFO - await self.middleware_stack(scope, receive, send) 2025-07-20 15:47:13,396 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/starlette/routing.py", line 734, in app 2025-07-20 15:47:13,396 - sglang - INFO - await route.handle(scope, receive, send) 2025-07-20 15:47:13,396 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/starlette/routing.py", line 288, in handle 2025-07-20 15:47:13,396 - sglang - INFO - await self.app(scope, receive, send) 2025-07-20 15:47:13,397 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/starlette/routing.py", line 76, in app 2025-07-20 15:47:13,397 - sglang - INFO - await wrap_app_handling_exceptions(app, request)(scope, receive, send) 2025-07-20 15:47:13,397 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app 2025-07-20 15:47:13,397 - sglang - INFO - raise exc 2025-07-20 15:47:13,397 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app 2025-07-20 15:47:13,397 - sglang - INFO - await app(scope, receive, sender) 2025-07-20 15:47:13,397 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/starlette/routing.py", line 73, in app 2025-07-20 15:47:13,397 - sglang - INFO - response = await f(request) 2025-07-20 15:47:13,397 - sglang - INFO - ^^^^^^^^^^^^^^^^ 2025-07-20 15:47:13,397 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/fastapi/routing.py", line 301, in app 2025-07-20 15:47:13,397 - sglang - INFO - raw_response = await run_endpoint_function( 2025-07-20 15:47:13,397 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 15:47:13,397 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/fastapi/routing.py", line 212, in run_endpoint_function 2025-07-20 15:47:13,397 - sglang - INFO - return await dependant.call(**values) 2025-07-20 15:47:13,397 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 15:47:13,398 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/entrypoints/http_server.py", line 406, in openai_v1_chat_completions 2025-07-20 15:47:13,398 - sglang - INFO - return await v1_chat_completions(_global_state.tokenizer_manager, raw_request) 2025-07-20 15:47:13,398 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 15:47:13,398 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/openai_api/adapter.py", line 1426, in v1_chat_completions 2025-07-20 15:47:13,398 - sglang - INFO - ret = await tokenizer_manager.generate_request( 2025-07-20 15:47:13,398 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 15:47:13,398 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/managers/tokenizer_manager.py", line 291, in generate_request 2025-07-20 15:47:13,398 - sglang - INFO - tokenized_obj = await self._tokenize_one_request(obj) 2025-07-20 15:47:13,398 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 15:47:13,398 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/managers/tokenizer_manager.py", line 331, in _tokenize_one_request 2025-07-20 15:47:13,398 - sglang - INFO - image_inputs: Dict = await self.image_processor.process_images_async( 2025-07-20 15:47:13,398 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 15:47:13,398 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/managers/image_processor.py", line 474, in process_images_async 2025-07-20 15:47:13,398 - sglang - INFO - pixel_values, image_hash, image_size, image_grid_thw = ( 2025-07-20 15:47:13,399 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 15:47:13,399 - sglang - INFO - TypeError: cannot unpack non-iterable NoneType object 2025-07-20 15:47:13,399 - __main__ - WARNING - ValueError on attempt 6 for tests/gnarly_pdfs/skinnypage.pdf-2: - Got InternalServerError from server: b'Internal Server Error', skipping this response 2025-07-20 15:47:13,758 - __main__ - INFO - Built page query for tests/gnarly_pdfs/skinnypage.pdf-2 2025-07-20 15:47:13,766 - sglang - INFO - [2025-07-20 15:47:13 TP0] Prefill batch. #new-seq: 1, #new-token: 1665, #cached-token: 0, cache hit rate: 0.00%, token usage: 0.68, #running-req: 8, #queue-req: 519 2025-07-20 15:47:13,766 - __main__ - INFO - sglang running req: 8 queue req: 519 2025-07-20 15:47:13,786 - sglang - INFO - [2025-07-20 15:47:13] Exception in TokenizerManager: 2025-07-20 15:47:13,786 - sglang - INFO - Traceback (most recent call last): 2025-07-20 15:47:13,786 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/managers/image_processor.py", line 417, in _process_single_image_task 2025-07-20 15:47:13,786 - sglang - INFO - process_result = image_processor(image) 2025-07-20 15:47:13,786 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 15:47:13,786 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/transformers/image_processing_utils.py", line 41, in __call__ 2025-07-20 15:47:13,786 - sglang - INFO - return self.preprocess(images, **kwargs) 2025-07-20 15:47:13,786 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 15:47:13,786 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/transformers/models/qwen2_vl/image_processing_qwen2_vl.py", line 417, in preprocess 2025-07-20 15:47:13,786 - sglang - INFO - patches, image_grid_thw = self._preprocess( 2025-07-20 15:47:13,786 - sglang - INFO - ^^^^^^^^^^^^^^^^^ 2025-07-20 15:47:13,786 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/transformers/models/qwen2_vl/image_processing_qwen2_vl.py", line 269, in _preprocess 2025-07-20 15:47:13,786 - sglang - INFO - resized_height, resized_width = smart_resize( 2025-07-20 15:47:13,787 - sglang - INFO - ^^^^^^^^^^^^^ 2025-07-20 15:47:13,787 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/transformers/models/qwen2_vl/image_processing_qwen2_vl.py", line 112, in smart_resize 2025-07-20 15:47:13,787 - sglang - INFO - raise ValueError(f"height:{height} or width:{width} must be larger than factor:{factor}") 2025-07-20 15:47:13,787 - sglang - INFO - ValueError: height:1024 or width:17 must be larger than factor:28 2025-07-20 15:47:13,787 - sglang - INFO - 2025-07-20 15:47:13,791 - sglang - INFO - [2025-07-20 15:47:13] ERROR: Exception in ASGI application 2025-07-20 15:47:13,791 - sglang - INFO - Traceback (most recent call last): 2025-07-20 15:47:13,791 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/uvicorn/protocols/http/httptools_impl.py", line 409, in run_asgi 2025-07-20 15:47:13,791 - sglang - INFO - result = await app( # type: ignore[func-returns-value] 2025-07-20 15:47:13,791 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 15:47:13,791 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/uvicorn/middleware/proxy_headers.py", line 60, in __call__ 2025-07-20 15:47:13,791 - sglang - INFO - return await self.app(scope, receive, send) 2025-07-20 15:47:13,791 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 15:47:13,791 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/fastapi/applications.py", line 1054, in __call__ 2025-07-20 15:47:13,791 - sglang - INFO - await super().__call__(scope, receive, send) 2025-07-20 15:47:13,792 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/starlette/applications.py", line 112, in __call__ 2025-07-20 15:47:13,792 - sglang - INFO - await self.middleware_stack(scope, receive, send) 2025-07-20 15:47:13,792 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/starlette/middleware/errors.py", line 187, in __call__ 2025-07-20 15:47:13,792 - sglang - INFO - raise exc 2025-07-20 15:47:13,792 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/starlette/middleware/errors.py", line 165, in __call__ 2025-07-20 15:47:13,792 - sglang - INFO - await self.app(scope, receive, _send) 2025-07-20 15:47:13,792 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/starlette/middleware/cors.py", line 85, in __call__ 2025-07-20 15:47:13,792 - sglang - INFO - await self.app(scope, receive, send) 2025-07-20 15:47:13,792 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/starlette/middleware/exceptions.py", line 62, in __call__ 2025-07-20 15:47:13,792 - sglang - INFO - await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send) 2025-07-20 15:47:13,792 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app 2025-07-20 15:47:13,792 - sglang - INFO - raise exc 2025-07-20 15:47:13,792 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app 2025-07-20 15:47:13,792 - sglang - INFO - await app(scope, receive, sender) 2025-07-20 15:47:13,792 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/starlette/routing.py", line 714, in __call__ 2025-07-20 15:47:13,793 - sglang - INFO - await self.middleware_stack(scope, receive, send) 2025-07-20 15:47:13,793 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/starlette/routing.py", line 734, in app 2025-07-20 15:47:13,793 - sglang - INFO - await route.handle(scope, receive, send) 2025-07-20 15:47:13,793 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/starlette/routing.py", line 288, in handle 2025-07-20 15:47:13,793 - sglang - INFO - await self.app(scope, receive, send) 2025-07-20 15:47:13,793 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/starlette/routing.py", line 76, in app 2025-07-20 15:47:13,793 - sglang - INFO - await wrap_app_handling_exceptions(app, request)(scope, receive, send) 2025-07-20 15:47:13,793 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app 2025-07-20 15:47:13,793 - sglang - INFO - raise exc 2025-07-20 15:47:13,793 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app 2025-07-20 15:47:13,793 - sglang - INFO - await app(scope, receive, sender) 2025-07-20 15:47:13,793 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/starlette/routing.py", line 73, in app 2025-07-20 15:47:13,793 - sglang - INFO - response = await f(request) 2025-07-20 15:47:13,793 - sglang - INFO - ^^^^^^^^^^^^^^^^ 2025-07-20 15:47:13,793 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/fastapi/routing.py", line 301, in app 2025-07-20 15:47:13,794 - sglang - INFO - raw_response = await run_endpoint_function( 2025-07-20 15:47:13,794 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 15:47:13,794 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/fastapi/routing.py", line 212, in run_endpoint_function 2025-07-20 15:47:13,794 - sglang - INFO - return await dependant.call(**values) 2025-07-20 15:47:13,794 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 15:47:13,794 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/entrypoints/http_server.py", line 406, in openai_v1_chat_completions 2025-07-20 15:47:13,794 - sglang - INFO - return await v1_chat_completions(_global_state.tokenizer_manager, raw_request) 2025-07-20 15:47:13,794 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 15:47:13,794 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/openai_api/adapter.py", line 1426, in v1_chat_completions 2025-07-20 15:47:13,794 - sglang - INFO - ret = await tokenizer_manager.generate_request( 2025-07-20 15:47:13,794 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 15:47:13,794 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/managers/tokenizer_manager.py", line 291, in generate_request 2025-07-20 15:47:13,794 - sglang - INFO - tokenized_obj = await self._tokenize_one_request(obj) 2025-07-20 15:47:13,794 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 15:47:13,794 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/managers/tokenizer_manager.py", line 331, in _tokenize_one_request 2025-07-20 15:47:13,794 - sglang - INFO - image_inputs: Dict = await self.image_processor.process_images_async( 2025-07-20 15:47:13,795 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 15:47:13,795 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/managers/image_processor.py", line 474, in process_images_async 2025-07-20 15:47:13,795 - sglang - INFO - pixel_values, image_hash, image_size, image_grid_thw = ( 2025-07-20 15:47:13,795 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 15:47:13,795 - sglang - INFO - TypeError: cannot unpack non-iterable NoneType object 2025-07-20 15:47:13,795 - __main__ - WARNING - ValueError on attempt 7 for tests/gnarly_pdfs/skinnypage.pdf-2: - Got InternalServerError from server: b'Internal Server Error', skipping this response 2025-07-20 15:47:13,796 - __main__ - ERROR - Failed to process tests/gnarly_pdfs/skinnypage.pdf-2 after 8 attempts. 2025-07-20 15:47:14,614 - sglang - INFO - [2025-07-20 15:47:14 TP0] Decode batch. #running-req: 9, #token: 27427, token usage: 0.72, gen throughput (token/s): 213.85, #queue-req: 519 2025-07-20 15:47:14,615 - __main__ - INFO - sglang running req: 9 queue req: 519 2025-07-20 15:47:14,745 - __main__ - INFO - Queue remaining: 0 2025-07-20 15:47:14,746 - __main__ - INFO - Metric Name Lifetime (tokens/sec) Recently (tokens/sec) ---------------------------------------------------------------------------------- finished_input_tokens 30.85 137.88 finished_output_tokens 11.83 52.86 sglang_input_tokens 904.76 888.36 sglang_output_tokens 259.71 263.92 2025-07-20 15:47:14,746 - __main__ - INFO - Worker ID | errored | finished | started ----------+---------+----------+-------- 0 | 0 | 497 | 500 1 | 0 | 10 | 10 2 | 0 | 5 | 5 3 | 1 | 2 | 529 2025-07-20 15:47:14,907 - sglang - INFO - [2025-07-20 15:47:14 TP0] Prefill batch. #new-seq: 1, #new-token: 2652, #cached-token: 0, cache hit rate: 0.00%, token usage: 0.67, #running-req: 8, #queue-req: 518 2025-07-20 15:47:14,907 - __main__ - INFO - sglang running req: 8 queue req: 518 2025-07-20 15:47:16,391 - sglang - INFO - [2025-07-20 15:47:16 TP0] Decode batch. #running-req: 9, #token: 28177, token usage: 0.74, gen throughput (token/s): 202.06, #queue-req: 518 2025-07-20 15:47:16,391 - __main__ - INFO - sglang running req: 9 queue req: 518 2025-07-20 15:47:16,430 - __main__ - INFO - Built page query for tests/gnarly_pdfs/map1.pdf-1 2025-07-20 15:47:16,788 - __main__ - WARNING - ValueError on attempt 1 for tests/gnarly_pdfs/map1.pdf-1: - Got BadRequestError from server: b'{"object":"error","message":"The input (78749 tokens) is longer than the model\'s context length (32768 tokens).","type":"BadRequestError","param":null,"code":400}', skipping this response 2025-07-20 15:47:17,385 - sglang - INFO - [2025-07-20 15:47:17 TP0] Decode batch. #running-req: 9, #token: 28537, token usage: 0.75, gen throughput (token/s): 362.07, #queue-req: 518 2025-07-20 15:47:17,386 - __main__ - INFO - sglang running req: 9 queue req: 518 2025-07-20 15:47:18,401 - sglang - INFO - [2025-07-20 15:47:18 TP0] Decode batch. #running-req: 9, #token: 28897, token usage: 0.76, gen throughput (token/s): 354.31, #queue-req: 518 2025-07-20 15:47:18,402 - __main__ - INFO - sglang running req: 9 queue req: 518 2025-07-20 15:47:19,510 - sglang - INFO - [2025-07-20 15:47:19 TP0] Decode batch. #running-req: 9, #token: 29257, token usage: 0.77, gen throughput (token/s): 324.66, #queue-req: 518 2025-07-20 15:47:19,511 - __main__ - INFO - sglang running req: 9 queue req: 518 2025-07-20 15:47:20,500 - sglang - INFO - [2025-07-20 15:47:20 TP0] Decode batch. #running-req: 9, #token: 29617, token usage: 0.78, gen throughput (token/s): 363.90, #queue-req: 518 2025-07-20 15:47:20,500 - __main__ - INFO - sglang running req: 9 queue req: 518 2025-07-20 15:47:21,480 - sglang - INFO - [2025-07-20 15:47:21 TP0] Decode batch. #running-req: 9, #token: 29977, token usage: 0.79, gen throughput (token/s): 367.29, #queue-req: 518 2025-07-20 15:47:21,480 - __main__ - INFO - sglang running req: 9 queue req: 518 2025-07-20 15:47:21,784 - __main__ - INFO - Built page query for tests/gnarly_pdfs/map1.pdf-1 2025-07-20 15:47:22,060 - __main__ - WARNING - ValueError on attempt 2 for tests/gnarly_pdfs/map1.pdf-1: - Got BadRequestError from server: b'{"object":"error","message":"The input (78749 tokens) is longer than the model\'s context length (32768 tokens).","type":"BadRequestError","param":null,"code":400}', skipping this response 2025-07-20 15:47:22,124 - __main__ - WARNING - JSON decode error on attempt 2 for scripts/data/11445200MB2C47380T4440125017008 (1).pdf-12: Unterminated string starting at: line 1 column 125 (char 124) 2025-07-20 15:47:22,142 - sglang - INFO - [2025-07-20 15:47:22 TP0] Prefill batch. #new-seq: 1, #new-token: 2744, #cached-token: 0, cache hit rate: 0.00%, token usage: 0.69, #running-req: 8, #queue-req: 517 2025-07-20 15:47:22,142 - __main__ - INFO - sglang running req: 8 queue req: 517 2025-07-20 15:47:22,402 - __main__ - INFO - Built page query for scripts/data/11445200MB2C47380T4440125017008 (1).pdf-12 2025-07-20 15:47:23,301 - sglang - INFO - [2025-07-20 15:47:23 TP0] Decode batch. #running-req: 9, #token: 29055, token usage: 0.76, gen throughput (token/s): 197.12, #queue-req: 518 2025-07-20 15:47:23,301 - __main__ - INFO - sglang running req: 9 queue req: 518 2025-07-20 15:47:24,101 - sglang - INFO - [2025-07-20 15:47:24 TP0] Prefill batch. #new-seq: 1, #new-token: 1677, #cached-token: 0, cache hit rate: 0.00%, token usage: 0.72, #running-req: 8, #queue-req: 517 2025-07-20 15:47:24,101 - __main__ - INFO - sglang running req: 8 queue req: 517 2025-07-20 15:47:24,747 - __main__ - INFO - Queue remaining: 0 2025-07-20 15:47:24,747 - __main__ - INFO - Metric Name Lifetime (tokens/sec) Recently (tokens/sec) ---------------------------------------------------------------------------------- finished_input_tokens 30.62 137.88 finished_output_tokens 11.74 52.86 sglang_input_tokens 902.88 869.90 sglang_output_tokens 259.10 259.87 2025-07-20 15:47:24,748 - __main__ - INFO - Worker ID | errored | finished | started ----------+---------+----------+-------- 0 | 0 | 497 | 500 1 | 0 | 10 | 10 2 | 0 | 5 | 5 3 | 1 | 4 | 529 2025-07-20 15:47:24,937 - sglang - INFO - [2025-07-20 15:47:24 TP0] Decode batch. #running-req: 9, #token: 29106, token usage: 0.77, gen throughput (token/s): 219.44, #queue-req: 517 2025-07-20 15:47:24,937 - __main__ - INFO - sglang running req: 9 queue req: 517 2025-07-20 15:47:25,318 - __main__ - WARNING - JSON decode error on attempt 2 for scripts/data/11445200MB2D6222364440125017008.pdf-13: Unterminated string starting at: line 1 column 125 (char 124) 2025-07-20 15:47:25,334 - sglang - INFO - [2025-07-20 15:47:25 TP0] Prefill batch. #new-seq: 2, #new-token: 5740, #cached-token: 0, cache hit rate: 0.00%, token usage: 0.61, #running-req: 8, #queue-req: 515 2025-07-20 15:47:25,334 - __main__ - INFO - sglang running req: 8 queue req: 515 2025-07-20 15:47:25,589 - __main__ - INFO - Built page query for scripts/data/11445200MB2D6222364440125017008.pdf-13 2025-07-20 15:47:27,163 - sglang - INFO - [2025-07-20 15:47:27 TP0] Prefill batch. #new-seq: 1, #new-token: 3205, #cached-token: 0, cache hit rate: 0.00%, token usage: 0.76, #running-req: 9, #queue-req: 515 2025-07-20 15:47:27,163 - __main__ - INFO - sglang running req: 9 queue req: 515 2025-07-20 15:47:27,285 - __main__ - INFO - Built page query for tests/gnarly_pdfs/map1.pdf-1 2025-07-20 15:47:27,533 - __main__ - WARNING - ValueError on attempt 3 for tests/gnarly_pdfs/map1.pdf-1: - Got BadRequestError from server: b'{"object":"error","message":"The input (78749 tokens) is longer than the model\'s context length (32768 tokens).","type":"BadRequestError","param":null,"code":400}', skipping this response 2025-07-20 15:47:28,698 - sglang - INFO - [2025-07-20 15:47:28 TP0] Decode batch. #running-req: 10, #token: 32321, token usage: 0.85, gen throughput (token/s): 101.58, #queue-req: 515 2025-07-20 15:47:28,698 - __main__ - INFO - sglang running req: 10 queue req: 515 2025-07-20 15:47:29,689 - sglang - INFO - [2025-07-20 15:47:29 TP0] Decode batch. #running-req: 10, #token: 32721, token usage: 0.86, gen throughput (token/s): 403.49, #queue-req: 515 2025-07-20 15:47:29,689 - __main__ - INFO - sglang running req: 10 queue req: 515 2025-07-20 15:47:30,211 - sglang - INFO - [2025-07-20 15:47:30 TP0] Prefill batch. #new-seq: 1, #new-token: 2912, #cached-token: 0, cache hit rate: 0.00%, token usage: 0.75, #running-req: 9, #queue-req: 514 2025-07-20 15:47:30,211 - __main__ - INFO - sglang running req: 9 queue req: 514 2025-07-20 15:47:31,546 - sglang - INFO - [2025-07-20 15:47:31 TP0] Decode batch. #running-req: 10, #token: 31594, token usage: 0.83, gen throughput (token/s): 214.81, #queue-req: 514 2025-07-20 15:47:31,547 - __main__ - INFO - sglang running req: 10 queue req: 514 2025-07-20 15:47:32,494 - __main__ - INFO - Built page query for tests/gnarly_pdfs/map1.pdf-1 2025-07-20 15:47:32,667 - sglang - INFO - [2025-07-20 15:47:32 TP0] Decode batch. #running-req: 10, #token: 31994, token usage: 0.84, gen throughput (token/s): 357.05, #queue-req: 514 2025-07-20 15:47:32,667 - __main__ - INFO - sglang running req: 10 queue req: 514 2025-07-20 15:47:32,992 - __main__ - WARNING - ValueError on attempt 4 for tests/gnarly_pdfs/map1.pdf-1: - Got BadRequestError from server: b'{"object":"error","message":"The input (78749 tokens) is longer than the model\'s context length (32768 tokens).","type":"BadRequestError","param":null,"code":400}', skipping this response 2025-07-20 15:47:33,735 - sglang - INFO - [2025-07-20 15:47:33 TP0] Decode batch. #running-req: 10, #token: 32394, token usage: 0.85, gen throughput (token/s): 374.52, #queue-req: 514 2025-07-20 15:47:33,735 - __main__ - INFO - sglang running req: 10 queue req: 514 2025-07-20 15:47:34,749 - __main__ - INFO - Queue remaining: 0 2025-07-20 15:47:34,749 - __main__ - INFO - Metric Name Lifetime (tokens/sec) Recently (tokens/sec) ---------------------------------------------------------------------------------- finished_input_tokens 30.39 137.88 finished_output_tokens 11.65 52.86 sglang_input_tokens 903.53 859.63 sglang_output_tokens 259.07 258.80 2025-07-20 15:47:34,749 - __main__ - INFO - Worker ID | errored | finished | started ----------+---------+----------+-------- 0 | 0 | 497 | 500 1 | 0 | 10 | 10 2 | 0 | 5 | 5 3 | 1 | 7 | 529 2025-07-20 15:47:34,749 - sglang - INFO - [2025-07-20 15:47:34 TP0] Decode batch. #running-req: 9, #token: 30869, token usage: 0.81, gen throughput (token/s): 388.23, #queue-req: 514 2025-07-20 15:47:34,750 - __main__ - INFO - sglang running req: 9 queue req: 514 2025-07-20 15:47:35,732 - sglang - INFO - [2025-07-20 15:47:35 TP0] Decode batch. #running-req: 9, #token: 31229, token usage: 0.82, gen throughput (token/s): 366.41, #queue-req: 514 2025-07-20 15:47:35,732 - __main__ - INFO - sglang running req: 9 queue req: 514 2025-07-20 15:47:36,517 - sglang - INFO - [2025-07-20 15:47:36 TP0] Prefill batch. #new-seq: 1, #new-token: 2586, #cached-token: 0, cache hit rate: 0.00%, token usage: 0.71, #running-req: 8, #queue-req: 513 2025-07-20 15:47:36,518 - __main__ - INFO - sglang running req: 8 queue req: 513 2025-07-20 15:47:37,533 - sglang - INFO - [2025-07-20 15:47:37 TP0] Decode batch. #running-req: 9, #token: 29582, token usage: 0.78, gen throughput (token/s): 199.39, #queue-req: 513 2025-07-20 15:47:37,533 - __main__ - INFO - sglang running req: 9 queue req: 513 2025-07-20 15:47:37,828 - __main__ - INFO - Built page query for tests/gnarly_pdfs/map1.pdf-1 2025-07-20 15:47:38,074 - __main__ - WARNING - ValueError on attempt 5 for tests/gnarly_pdfs/map1.pdf-1: - Got BadRequestError from server: b'{"object":"error","message":"The input (78749 tokens) is longer than the model\'s context length (32768 tokens).","type":"BadRequestError","param":null,"code":400}', skipping this response 2025-07-20 15:47:38,519 - sglang - INFO - [2025-07-20 15:47:38 TP0] Decode batch. #running-req: 9, #token: 29942, token usage: 0.79, gen throughput (token/s): 365.13, #queue-req: 513 2025-07-20 15:47:38,519 - __main__ - INFO - sglang running req: 9 queue req: 513 2025-07-20 15:47:39,571 - sglang - INFO - [2025-07-20 15:47:39 TP0] Decode batch. #running-req: 9, #token: 30302, token usage: 0.80, gen throughput (token/s): 342.16, #queue-req: 513 2025-07-20 15:47:39,571 - __main__ - INFO - sglang running req: 9 queue req: 513 2025-07-20 15:47:39,794 - sglang - INFO - [2025-07-20 15:47:39 TP0] Prefill batch. #new-seq: 1, #new-token: 2147, #cached-token: 0, cache hit rate: 0.00%, token usage: 0.68, #running-req: 8, #queue-req: 512 2025-07-20 15:47:39,794 - __main__ - INFO - sglang running req: 8 queue req: 512 2025-07-20 15:47:41,477 - sglang - INFO - [2025-07-20 15:47:41 TP0] Decode batch. #running-req: 9, #token: 28236, token usage: 0.74, gen throughput (token/s): 188.38, #queue-req: 512 2025-07-20 15:47:41,477 - __main__ - INFO - sglang running req: 9 queue req: 512 2025-07-20 15:47:42,135 - sglang - INFO - [2025-07-20 15:47:42 TP0] Prefill batch. #new-seq: 1, #new-token: 3094, #cached-token: 0, cache hit rate: 0.00%, token usage: 0.66, #running-req: 8, #queue-req: 511 2025-07-20 15:47:42,135 - __main__ - INFO - sglang running req: 8 queue req: 511 2025-07-20 15:47:43,397 - sglang - INFO - [2025-07-20 15:47:43 TP0] Decode batch. #running-req: 9, #token: 28425, token usage: 0.75, gen throughput (token/s): 186.91, #queue-req: 511 2025-07-20 15:47:43,398 - __main__ - INFO - sglang running req: 9 queue req: 511 2025-07-20 15:47:43,614 - __main__ - INFO - Built page query for tests/gnarly_pdfs/map1.pdf-1 2025-07-20 15:47:43,949 - __main__ - WARNING - ValueError on attempt 6 for tests/gnarly_pdfs/map1.pdf-1: - Got BadRequestError from server: b'{"object":"error","message":"The input (78749 tokens) is longer than the model\'s context length (32768 tokens).","type":"BadRequestError","param":null,"code":400}', skipping this response 2025-07-20 15:47:44,382 - sglang - INFO - [2025-07-20 15:47:44 TP0] Decode batch. #running-req: 9, #token: 28785, token usage: 0.76, gen throughput (token/s): 365.76, #queue-req: 511 2025-07-20 15:47:44,382 - __main__ - INFO - sglang running req: 9 queue req: 511 2025-07-20 15:47:44,750 - __main__ - INFO - Queue remaining: 0 2025-07-20 15:47:44,750 - __main__ - INFO - Metric Name Lifetime (tokens/sec) Recently (tokens/sec) ---------------------------------------------------------------------------------- finished_input_tokens 30.17 137.88 finished_output_tokens 11.57 52.86 sglang_input_tokens 904.02 870.38 sglang_output_tokens 259.17 262.51 2025-07-20 15:47:44,750 - __main__ - INFO - Worker ID | errored | finished | started ----------+---------+----------+-------- 0 | 0 | 497 | 500 1 | 0 | 10 | 10 2 | 0 | 5 | 5 3 | 1 | 10 | 529 2025-07-20 15:47:45,371 - sglang - INFO - [2025-07-20 15:47:45 TP0] Decode batch. #running-req: 9, #token: 29145, token usage: 0.77, gen throughput (token/s): 363.94, #queue-req: 511 2025-07-20 15:47:45,371 - __main__ - INFO - sglang running req: 9 queue req: 511 2025-07-20 15:47:46,355 - sglang - INFO - [2025-07-20 15:47:46 TP0] Decode batch. #running-req: 9, #token: 29505, token usage: 0.78, gen throughput (token/s): 365.85, #queue-req: 511 2025-07-20 15:47:46,355 - __main__ - INFO - sglang running req: 9 queue req: 511 2025-07-20 15:47:47,338 - sglang - INFO - [2025-07-20 15:47:47 TP0] Decode batch. #running-req: 9, #token: 29865, token usage: 0.79, gen throughput (token/s): 366.14, #queue-req: 511 2025-07-20 15:47:47,338 - __main__ - INFO - sglang running req: 9 queue req: 511 2025-07-20 15:47:48,319 - sglang - INFO - [2025-07-20 15:47:48 TP0] Decode batch. #running-req: 9, #token: 30225, token usage: 0.80, gen throughput (token/s): 366.86, #queue-req: 511 2025-07-20 15:47:48,320 - __main__ - INFO - sglang running req: 9 queue req: 511 2025-07-20 15:47:48,446 - __main__ - INFO - Built page query for tests/gnarly_pdfs/map1.pdf-1 2025-07-20 15:47:48,734 - __main__ - WARNING - ValueError on attempt 7 for tests/gnarly_pdfs/map1.pdf-1: - Got BadRequestError from server: b'{"object":"error","message":"The input (78749 tokens) is longer than the model\'s context length (32768 tokens).","type":"BadRequestError","param":null,"code":400}', skipping this response 2025-07-20 15:47:48,735 - __main__ - ERROR - Failed to process tests/gnarly_pdfs/map1.pdf-1 after 8 attempts. 2025-07-20 15:47:49,112 - __main__ - ERROR - Document tests/gnarly_pdfs/map1.pdf has 1 fallback pages out of 1 exceeding max_page_error_rate of 0.004, discarding document. 2025-07-20 15:47:49,113 - sglang - INFO - [2025-07-20 15:47:48 TP0] Prefill batch. #new-seq: 1, #new-token: 3020, #cached-token: 0, cache hit rate: 0.00%, token usage: 0.70, #running-req: 8, #queue-req: 510 2025-07-20 15:47:49,113 - __main__ - INFO - sglang running req: 8 queue req: 510 2025-07-20 15:47:50,207 - sglang - INFO - [2025-07-20 15:47:50 TP0] Decode batch. #running-req: 9, #token: 29711, token usage: 0.78, gen throughput (token/s): 190.20, #queue-req: 510 2025-07-20 15:47:50,207 - __main__ - INFO - sglang running req: 9 queue req: 510 2025-07-20 15:47:50,304 - sglang - INFO - [2025-07-20 15:47:50 TP0] Prefill batch. #new-seq: 1, #new-token: 2297, #cached-token: 0, cache hit rate: 0.00%, token usage: 0.69, #running-req: 8, #queue-req: 509 2025-07-20 15:47:50,304 - __main__ - INFO - sglang running req: 8 queue req: 509 2025-07-20 15:47:51,308 - sglang - INFO - [2025-07-20 15:47:51 TP0] Prefill batch. #new-seq: 2, #new-token: 4138, #cached-token: 0, cache hit rate: 0.00%, token usage: 0.66, #running-req: 8, #queue-req: 507 2025-07-20 15:47:51,309 - __main__ - INFO - sglang running req: 8 queue req: 507 2025-07-20 15:47:53,095 - sglang - INFO - [2025-07-20 15:47:53 TP0] Prefill batch. #new-seq: 1, #new-token: 2902, #cached-token: 0, cache hit rate: 0.00%, token usage: 0.67, #running-req: 9, #queue-req: 506 2025-07-20 15:47:53,095 - __main__ - INFO - sglang running req: 9 queue req: 506 2025-07-20 15:47:54,208 - sglang - INFO - [2025-07-20 15:47:54 TP0] Decode batch. #running-req: 10, #token: 28558, token usage: 0.75, gen throughput (token/s): 95.72, #queue-req: 506 2025-07-20 15:47:54,208 - __main__ - INFO - sglang running req: 10 queue req: 506 2025-07-20 15:47:54,429 - sglang - INFO - [2025-07-20 15:47:54 TP0] Prefill batch. #new-seq: 1, #new-token: 2889, #cached-token: 0, cache hit rate: 0.00%, token usage: 0.67, #running-req: 9, #queue-req: 505 2025-07-20 15:47:54,430 - __main__ - INFO - sglang running req: 9 queue req: 505 2025-07-20 15:47:54,751 - __main__ - INFO - Queue remaining: 0 2025-07-20 15:47:54,751 - __main__ - INFO - Metric Name Lifetime (tokens/sec) Recently (tokens/sec) ---------------------------------------------------------------------------------- finished_input_tokens 29.95 137.88 finished_output_tokens 11.48 52.86 sglang_input_tokens 907.75 880.16 sglang_output_tokens 259.92 263.96 2025-07-20 15:47:54,751 - __main__ - INFO - Worker ID | errored | finished | started ----------+---------+----------+-------- 0 | 0 | 497 | 500 1 | 0 | 10 | 10 2 | 0 | 5 | 5 3 | 2 | 15 | 529 2025-07-20 15:47:56,021 - sglang - INFO - [2025-07-20 15:47:56 TP0] Decode batch. #running-req: 10, #token: 28804, token usage: 0.76, gen throughput (token/s): 220.13, #queue-req: 505 2025-07-20 15:47:56,021 - __main__ - INFO - sglang running req: 10 queue req: 505 2025-07-20 15:47:57,004 - sglang - INFO - [2025-07-20 15:47:57 TP0] Decode batch. #running-req: 10, #token: 29204, token usage: 0.77, gen throughput (token/s): 406.94, #queue-req: 505 2025-07-20 15:47:57,004 - __main__ - INFO - sglang running req: 10 queue req: 505 2025-07-20 15:47:57,423 - sglang - INFO - [2025-07-20 15:47:57 TP0] Prefill batch. #new-seq: 1, #new-token: 2650, #cached-token: 0, cache hit rate: 0.00%, token usage: 0.70, #running-req: 9, #queue-req: 504 2025-07-20 15:47:57,423 - __main__ - INFO - sglang running req: 9 queue req: 504 2025-07-20 15:47:58,772 - sglang - INFO - [2025-07-20 15:47:58 TP0] Decode batch. #running-req: 10, #token: 29658, token usage: 0.78, gen throughput (token/s): 225.56, #queue-req: 504 2025-07-20 15:47:58,773 - __main__ - INFO - sglang running req: 10 queue req: 504 2025-07-20 15:47:58,944 - sglang - INFO - [2025-07-20 15:47:58 TP0] Prefill batch. #new-seq: 1, #new-token: 2834, #cached-token: 0, cache hit rate: 0.00%, token usage: 0.69, #running-req: 9, #queue-req: 503 2025-07-20 15:47:58,945 - __main__ - INFO - sglang running req: 9 queue req: 503 2025-07-20 15:48:00,615 - sglang - INFO - [2025-07-20 15:48:00 TP0] Decode batch. #running-req: 10, #token: 29194, token usage: 0.77, gen throughput (token/s): 216.57, #queue-req: 503 2025-07-20 15:48:00,615 - __main__ - INFO - sglang running req: 10 queue req: 503 2025-07-20 15:48:01,599 - sglang - INFO - [2025-07-20 15:48:01 TP0] Decode batch. #running-req: 10, #token: 29594, token usage: 0.78, gen throughput (token/s): 406.33, #queue-req: 503 2025-07-20 15:48:01,600 - __main__ - INFO - sglang running req: 10 queue req: 503 2025-07-20 15:48:02,583 - sglang - INFO - [2025-07-20 15:48:02 TP0] Decode batch. #running-req: 10, #token: 29994, token usage: 0.79, gen throughput (token/s): 406.46, #queue-req: 503 2025-07-20 15:48:02,584 - __main__ - INFO - sglang running req: 10 queue req: 503 2025-07-20 15:48:03,397 - sglang - INFO - [2025-07-20 15:48:03 TP0] Prefill batch. #new-seq: 1, #new-token: 3545, #cached-token: 0, cache hit rate: 0.00%, token usage: 0.74, #running-req: 9, #queue-req: 502 2025-07-20 15:48:03,397 - __main__ - INFO - sglang running req: 9 queue req: 502 2025-07-20 15:48:04,574 - sglang - INFO - [2025-07-20 15:48:04 TP0] Decode batch. #running-req: 10, #token: 31867, token usage: 0.84, gen throughput (token/s): 200.43, #queue-req: 502 2025-07-20 15:48:04,574 - __main__ - INFO - sglang running req: 10 queue req: 502 2025-07-20 15:48:04,752 - __main__ - INFO - Queue remaining: 0 2025-07-20 15:48:04,753 - __main__ - INFO - Metric Name Lifetime (tokens/sec) Recently (tokens/sec) ---------------------------------------------------------------------------------- finished_input_tokens 29.74 137.88 finished_output_tokens 11.40 52.86 sglang_input_tokens 906.13 863.87 sglang_output_tokens 259.15 257.14 2025-07-20 15:48:04,753 - __main__ - INFO - Worker ID | errored | finished | started ----------+---------+----------+-------- 0 | 0 | 497 | 500 1 | 0 | 10 | 10 2 | 0 | 5 | 5 3 | 2 | 18 | 529 2025-07-20 15:48:05,563 - sglang - INFO - [2025-07-20 15:48:05 TP0] Decode batch. #running-req: 10, #token: 32267, token usage: 0.85, gen throughput (token/s): 404.40, #queue-req: 502 2025-07-20 15:48:05,563 - __main__ - INFO - sglang running req: 10 queue req: 502 2025-07-20 15:48:06,280 - sglang - INFO - [2025-07-20 15:48:06 TP0] Prefill batch. #new-seq: 1, #new-token: 1258, #cached-token: 0, cache hit rate: 0.00%, token usage: 0.75, #running-req: 9, #queue-req: 501 2025-07-20 15:48:06,280 - __main__ - INFO - sglang running req: 9 queue req: 501 2025-07-20 15:48:07,091 - sglang - INFO - [2025-07-20 15:48:07 TP0] Decode batch. #running-req: 10, #token: 30019, token usage: 0.79, gen throughput (token/s): 261.12, #queue-req: 501 2025-07-20 15:48:07,092 - __main__ - INFO - sglang running req: 10 queue req: 501 2025-07-20 15:48:07,668 - sglang - INFO - [2025-07-20 15:48:07 TP0] Prefill batch. #new-seq: 1, #new-token: 2419, #cached-token: 0, cache hit rate: 0.00%, token usage: 0.76, #running-req: 9, #queue-req: 500 2025-07-20 15:48:07,668 - __main__ - INFO - sglang running req: 9 queue req: 500 2025-07-20 15:48:08,688 - sglang - INFO - [2025-07-20 15:48:08 TP0] Prefill batch. #new-seq: 1, #new-token: 2514, #cached-token: 0, cache hit rate: 0.00%, token usage: 0.73, #running-req: 9, #queue-req: 499 2025-07-20 15:48:08,689 - __main__ - INFO - sglang running req: 9 queue req: 499 2025-07-20 15:48:09,650 - sglang - INFO - [2025-07-20 15:48:09 TP0] Decode batch. #running-req: 10, #token: 30279, token usage: 0.80, gen throughput (token/s): 155.53, #queue-req: 499 2025-07-20 15:48:09,651 - __main__ - INFO - sglang running req: 10 queue req: 499 2025-07-20 15:48:10,637 - sglang - INFO - [2025-07-20 15:48:10 TP0] Decode batch. #running-req: 10, #token: 30679, token usage: 0.81, gen throughput (token/s): 405.59, #queue-req: 499 2025-07-20 15:48:10,637 - __main__ - INFO - sglang running req: 10 queue req: 499 2025-07-20 15:48:11,623 - sglang - INFO - [2025-07-20 15:48:11 TP0] Decode batch. #running-req: 10, #token: 31079, token usage: 0.82, gen throughput (token/s): 405.54, #queue-req: 499 2025-07-20 15:48:11,623 - __main__ - INFO - sglang running req: 10 queue req: 499 2025-07-20 15:48:12,611 - sglang - INFO - [2025-07-20 15:48:12 TP0] Decode batch. #running-req: 10, #token: 31479, token usage: 0.83, gen throughput (token/s): 404.73, #queue-req: 499 2025-07-20 15:48:12,611 - __main__ - INFO - sglang running req: 10 queue req: 499 2025-07-20 15:48:13,604 - sglang - INFO - [2025-07-20 15:48:13 TP0] Decode batch. #running-req: 10, #token: 31879, token usage: 0.84, gen throughput (token/s): 402.98, #queue-req: 499 2025-07-20 15:48:13,604 - __main__ - INFO - sglang running req: 10 queue req: 499 2025-07-20 15:48:14,147 - sglang - INFO - [2025-07-20 15:48:14 TP0] Prefill batch. #new-seq: 1, #new-token: 2365, #cached-token: 0, cache hit rate: 0.00%, token usage: 0.77, #running-req: 9, #queue-req: 498 2025-07-20 15:48:14,148 - __main__ - INFO - sglang running req: 9 queue req: 498 2025-07-20 15:48:14,754 - __main__ - INFO - Queue remaining: 0 2025-07-20 15:48:14,754 - __main__ - INFO - Metric Name Lifetime (tokens/sec) Recently (tokens/sec) ---------------------------------------------------------------------------------- finished_input_tokens 29.53 137.88 finished_output_tokens 11.32 52.86 sglang_input_tokens 906.42 855.53 sglang_output_tokens 259.08 254.82 2025-07-20 15:48:14,754 - __main__ - INFO - Worker ID | errored | finished | started ----------+---------+----------+-------- 0 | 0 | 497 | 500 1 | 0 | 10 | 10 2 | 0 | 5 | 5 3 | 2 | 22 | 529 2025-07-20 15:48:14,978 - sglang - INFO - [2025-07-20 15:48:14 TP0] Prefill batch. #new-seq: 1, #new-token: 2442, #cached-token: 0, cache hit rate: 0.00%, token usage: 0.75, #running-req: 9, #queue-req: 497 2025-07-20 15:48:14,978 - __main__ - INFO - sglang running req: 9 queue req: 497 2025-07-20 15:48:16,188 - sglang - INFO - [2025-07-20 15:48:16 TP0] Decode batch. #running-req: 10, #token: 31196, token usage: 0.82, gen throughput (token/s): 154.00, #queue-req: 497 2025-07-20 15:48:16,189 - __main__ - INFO - sglang running req: 10 queue req: 497 2025-07-20 15:48:17,184 - sglang - INFO - [2025-07-20 15:48:17 TP0] Decode batch. #running-req: 10, #token: 31596, token usage: 0.83, gen throughput (token/s): 401.78, #queue-req: 497 2025-07-20 15:48:17,184 - __main__ - INFO - sglang running req: 10 queue req: 497 2025-07-20 15:48:18,181 - sglang - INFO - [2025-07-20 15:48:18 TP0] Decode batch. #running-req: 10, #token: 31996, token usage: 0.84, gen throughput (token/s): 401.23, #queue-req: 497 2025-07-20 15:48:18,181 - __main__ - INFO - sglang running req: 10 queue req: 497 2025-07-20 15:48:18,256 - sglang - INFO - [2025-07-20 15:48:18 TP0] Prefill batch. #new-seq: 1, #new-token: 2877, #cached-token: 0, cache hit rate: 0.00%, token usage: 0.74, #running-req: 9, #queue-req: 496 2025-07-20 15:48:18,256 - __main__ - INFO - sglang running req: 9 queue req: 496 2025-07-20 15:48:19,841 - sglang - INFO - [2025-07-20 15:48:19 TP0] Prefill batch. #new-seq: 1, #new-token: 2407, #cached-token: 0, cache hit rate: 0.00%, token usage: 0.73, #running-req: 9, #queue-req: 495 2025-07-20 15:48:19,842 - __main__ - INFO - sglang running req: 9 queue req: 495 2025-07-20 15:48:20,807 - sglang - INFO - [2025-07-20 15:48:20 TP0] Decode batch. #running-req: 10, #token: 30299, token usage: 0.80, gen throughput (token/s): 151.57, #queue-req: 495 2025-07-20 15:48:20,807 - __main__ - INFO - sglang running req: 10 queue req: 495 2025-07-20 15:48:21,791 - sglang - INFO - [2025-07-20 15:48:21 TP0] Decode batch. #running-req: 10, #token: 30699, token usage: 0.81, gen throughput (token/s): 406.49, #queue-req: 495 2025-07-20 15:48:21,791 - __main__ - INFO - sglang running req: 10 queue req: 495 2025-07-20 15:48:22,782 - sglang - INFO - [2025-07-20 15:48:22 TP0] Decode batch. #running-req: 10, #token: 31099, token usage: 0.82, gen throughput (token/s): 403.58, #queue-req: 495 2025-07-20 15:48:22,782 - __main__ - INFO - sglang running req: 10 queue req: 495 2025-07-20 15:48:22,906 - sglang - INFO - [2025-07-20 15:48:22 TP0] Prefill batch. #new-seq: 1, #new-token: 2703, #cached-token: 0, cache hit rate: 0.00%, token usage: 0.73, #running-req: 9, #queue-req: 494 2025-07-20 15:48:22,906 - __main__ - INFO - sglang running req: 9 queue req: 494 2025-07-20 15:48:24,603 - sglang - INFO - [2025-07-20 15:48:24 TP0] Decode batch. #running-req: 10, #token: 30690, token usage: 0.81, gen throughput (token/s): 219.08, #queue-req: 494 2025-07-20 15:48:24,603 - __main__ - INFO - sglang running req: 10 queue req: 494 2025-07-20 15:48:24,756 - __main__ - INFO - Queue remaining: 0 2025-07-20 15:48:24,756 - __main__ - INFO - Metric Name Lifetime (tokens/sec) Recently (tokens/sec) ---------------------------------------------------------------------------------- finished_input_tokens 29.32 137.88 finished_output_tokens 11.24 52.86 sglang_input_tokens 907.82 860.97 sglang_output_tokens 259.20 255.92 2025-07-20 15:48:24,756 - __main__ - INFO - Worker ID | errored | finished | started ----------+---------+----------+-------- 0 | 0 | 497 | 500 1 | 0 | 10 | 10 2 | 0 | 5 | 5 3 | 2 | 26 | 529 2025-07-20 15:48:25,589 - sglang - INFO - [2025-07-20 15:48:25 TP0] Decode batch. #running-req: 10, #token: 31090, token usage: 0.82, gen throughput (token/s): 405.84, #queue-req: 494 2025-07-20 15:48:25,589 - __main__ - INFO - sglang running req: 10 queue req: 494 2025-07-20 15:48:26,576 - sglang - INFO - [2025-07-20 15:48:26 TP0] Decode batch. #running-req: 10, #token: 31490, token usage: 0.83, gen throughput (token/s): 405.25, #queue-req: 494 2025-07-20 15:48:26,576 - __main__ - INFO - sglang running req: 10 queue req: 494 2025-07-20 15:48:27,095 - sglang - INFO - [2025-07-20 15:48:27 TP0] Prefill batch. #new-seq: 1, #new-token: 3260, #cached-token: 0, cache hit rate: 0.00%, token usage: 0.75, #running-req: 9, #queue-req: 493 2025-07-20 15:48:27,095 - __main__ - INFO - sglang running req: 9 queue req: 493 2025-07-20 15:48:28,503 - sglang - INFO - [2025-07-20 15:48:28 TP0] Decode batch. #running-req: 9, #token: 29092, token usage: 0.77, gen throughput (token/s): 205.51, #queue-req: 493 2025-07-20 15:48:28,503 - __main__ - INFO - sglang running req: 9 queue req: 493 2025-07-20 15:48:29,477 - sglang - INFO - [2025-07-20 15:48:29 TP0] Decode batch. #running-req: 9, #token: 29452, token usage: 0.78, gen throughput (token/s): 369.34, #queue-req: 493 2025-07-20 15:48:29,478 - __main__ - INFO - sglang running req: 9 queue req: 493 2025-07-20 15:48:30,519 - sglang - INFO - [2025-07-20 15:48:30 TP0] Decode batch. #running-req: 9, #token: 29812, token usage: 0.78, gen throughput (token/s): 345.47, #queue-req: 493 2025-07-20 15:48:30,520 - __main__ - INFO - sglang running req: 9 queue req: 493 2025-07-20 15:48:31,509 - sglang - INFO - [2025-07-20 15:48:31 TP0] Decode batch. #running-req: 9, #token: 30172, token usage: 0.79, gen throughput (token/s): 363.61, #queue-req: 493 2025-07-20 15:48:31,510 - __main__ - INFO - sglang running req: 9 queue req: 493 2025-07-20 15:48:31,631 - sglang - INFO - [2025-07-20 15:48:31 TP0] Prefill batch. #new-seq: 1, #new-token: 3887, #cached-token: 0, cache hit rate: 0.00%, token usage: 0.68, #running-req: 8, #queue-req: 492 2025-07-20 15:48:31,632 - __main__ - INFO - sglang running req: 8 queue req: 492 2025-07-20 15:48:33,025 - sglang - INFO - [2025-07-20 15:48:33 TP0] Prefill batch. #new-seq: 1, #new-token: 2977, #cached-token: 0, cache hit rate: 0.00%, token usage: 0.71, #running-req: 8, #queue-req: 491 2025-07-20 15:48:33,025 - __main__ - INFO - sglang running req: 8 queue req: 491 2025-07-20 15:48:34,466 - sglang - INFO - [2025-07-20 15:48:34 TP0] Decode batch. #running-req: 9, #token: 30158, token usage: 0.79, gen throughput (token/s): 121.10, #queue-req: 491 2025-07-20 15:48:34,466 - __main__ - INFO - sglang running req: 9 queue req: 491 2025-07-20 15:48:34,757 - __main__ - INFO - Queue remaining: 0 2025-07-20 15:48:34,757 - __main__ - INFO - Metric Name Lifetime (tokens/sec) Recently (tokens/sec) ---------------------------------------------------------------------------------- finished_input_tokens 29.11 137.88 finished_output_tokens 11.16 52.86 sglang_input_tokens 909.07 861.92 sglang_output_tokens 259.10 256.07 2025-07-20 15:48:34,757 - __main__ - INFO - Worker ID | errored | finished | started ----------+---------+----------+-------- 0 | 0 | 497 | 500 1 | 0 | 10 | 10 2 | 0 | 5 | 5 3 | 2 | 30 | 529 2025-07-20 15:48:35,103 - sglang - INFO - [2025-07-20 15:48:35 TP0] Prefill batch. #new-seq: 1, #new-token: 2152, #cached-token: 0, cache hit rate: 0.00%, token usage: 0.72, #running-req: 8, #queue-req: 490 2025-07-20 15:48:35,104 - __main__ - INFO - sglang running req: 8 queue req: 490 2025-07-20 15:48:36,154 - sglang - INFO - [2025-07-20 15:48:36 TP0] Decode batch. #running-req: 9, #token: 29666, token usage: 0.78, gen throughput (token/s): 212.67, #queue-req: 490 2025-07-20 15:48:36,154 - __main__ - INFO - sglang running req: 9 queue req: 490 2025-07-20 15:48:37,131 - sglang - INFO - [2025-07-20 15:48:37 TP0] Decode batch. #running-req: 9, #token: 30026, token usage: 0.79, gen throughput (token/s): 368.24, #queue-req: 490 2025-07-20 15:48:37,132 - __main__ - INFO - sglang running req: 9 queue req: 490 2025-07-20 15:48:37,841 - sglang - INFO - [2025-07-20 15:48:37 TP0] Prefill batch. #new-seq: 1, #new-token: 2571, #cached-token: 0, cache hit rate: 0.00%, token usage: 0.69, #running-req: 8, #queue-req: 489 2025-07-20 15:48:37,841 - __main__ - INFO - sglang running req: 8 queue req: 489 2025-07-20 15:48:38,958 - sglang - INFO - [2025-07-20 15:48:38 TP0] Decode batch. #running-req: 9, #token: 28848, token usage: 0.76, gen throughput (token/s): 196.57, #queue-req: 489 2025-07-20 15:48:38,958 - __main__ - INFO - sglang running req: 9 queue req: 489 2025-07-20 15:48:39,006 - sglang - INFO - [2025-07-20 15:48:39 TP0] Prefill batch. #new-seq: 2, #new-token: 3412, #cached-token: 0, cache hit rate: 0.00%, token usage: 0.66, #running-req: 8, #queue-req: 487 2025-07-20 15:48:39,006 - __main__ - INFO - sglang running req: 8 queue req: 487 2025-07-20 15:48:40,664 - sglang - INFO - [2025-07-20 15:48:40 TP0] Prefill batch. #new-seq: 1, #new-token: 2399, #cached-token: 0, cache hit rate: 0.00%, token usage: 0.68, #running-req: 9, #queue-req: 486 2025-07-20 15:48:40,664 - __main__ - INFO - sglang running req: 9 queue req: 486 2025-07-20 15:48:41,961 - sglang - INFO - [2025-07-20 15:48:41 TP0] Decode batch. #running-req: 9, #token: 24834, token usage: 0.65, gen throughput (token/s): 131.52, #queue-req: 486 2025-07-20 15:48:41,961 - __main__ - INFO - sglang running req: 9 queue req: 486 2025-07-20 15:48:41,961 - sglang - INFO - [2025-07-20 15:48:41 TP0] Prefill batch. #new-seq: 1, #new-token: 3680, #cached-token: 0, cache hit rate: 0.00%, token usage: 0.65, #running-req: 9, #queue-req: 485 2025-07-20 15:48:41,961 - __main__ - INFO - sglang running req: 9 queue req: 485 2025-07-20 15:48:43,582 - sglang - INFO - [2025-07-20 15:48:43 TP0] Prefill batch. #new-seq: 1, #new-token: 2449, #cached-token: 0, cache hit rate: 0.00%, token usage: 0.72, #running-req: 9, #queue-req: 484 2025-07-20 15:48:43,582 - __main__ - INFO - sglang running req: 9 queue req: 484 2025-07-20 15:48:44,619 - sglang - INFO - [2025-07-20 15:48:44 TP0] Prefill batch. #new-seq: 1, #new-token: 3605, #cached-token: 0, cache hit rate: 0.00%, token usage: 0.70, #running-req: 9, #queue-req: 483 2025-07-20 15:48:44,619 - __main__ - INFO - sglang running req: 9 queue req: 483 2025-07-20 15:48:44,758 - __main__ - INFO - Queue remaining: 0 2025-07-20 15:48:44,758 - __main__ - INFO - Metric Name Lifetime (tokens/sec) Recently (tokens/sec) ---------------------------------------------------------------------------------- finished_input_tokens 28.91 137.88 finished_output_tokens 11.08 52.86 sglang_input_tokens 914.85 875.84 sglang_output_tokens 260.53 261.63 2025-07-20 15:48:44,758 - __main__ - INFO - Worker ID | errored | finished | started ----------+---------+----------+-------- 0 | 0 | 497 | 500 1 | 0 | 10 | 10 2 | 0 | 5 | 5 3 | 2 | 37 | 529 2025-07-20 15:48:45,792 - sglang - INFO - [2025-07-20 15:48:45 TP0] Decode batch. #running-req: 10, #token: 30298, token usage: 0.80, gen throughput (token/s): 103.89, #queue-req: 483 2025-07-20 15:48:45,792 - __main__ - INFO - sglang running req: 10 queue req: 483 2025-07-20 15:48:46,779 - sglang - INFO - [2025-07-20 15:48:46 TP0] Decode batch. #running-req: 10, #token: 30698, token usage: 0.81, gen throughput (token/s): 405.36, #queue-req: 483 2025-07-20 15:48:46,779 - __main__ - INFO - sglang running req: 10 queue req: 483 2025-07-20 15:48:47,766 - sglang - INFO - [2025-07-20 15:48:47 TP0] Decode batch. #running-req: 10, #token: 31098, token usage: 0.82, gen throughput (token/s): 405.24, #queue-req: 483 2025-07-20 15:48:47,766 - __main__ - INFO - sglang running req: 10 queue req: 483 2025-07-20 15:48:48,755 - sglang - INFO - [2025-07-20 15:48:48 TP0] Decode batch. #running-req: 10, #token: 31498, token usage: 0.83, gen throughput (token/s): 404.35, #queue-req: 483 2025-07-20 15:48:48,756 - __main__ - INFO - sglang running req: 10 queue req: 483 2025-07-20 15:48:49,746 - sglang - INFO - [2025-07-20 15:48:49 TP0] Decode batch. #running-req: 10, #token: 31898, token usage: 0.84, gen throughput (token/s): 403.72, #queue-req: 483 2025-07-20 15:48:49,746 - __main__ - INFO - sglang running req: 10 queue req: 483 2025-07-20 15:48:50,737 - sglang - INFO - [2025-07-20 15:48:50 TP0] Decode batch. #running-req: 10, #token: 32298, token usage: 0.85, gen throughput (token/s): 403.84, #queue-req: 483 2025-07-20 15:48:50,737 - __main__ - INFO - sglang running req: 10 queue req: 483 2025-07-20 15:48:51,725 - sglang - INFO - [2025-07-20 15:48:51 TP0] Decode batch. #running-req: 10, #token: 32698, token usage: 0.86, gen throughput (token/s): 404.52, #queue-req: 483 2025-07-20 15:48:51,727 - __main__ - INFO - sglang running req: 10 queue req: 483 2025-07-20 15:48:52,717 - sglang - INFO - [2025-07-20 15:48:52 TP0] Decode batch. #running-req: 10, #token: 33098, token usage: 0.87, gen throughput (token/s): 403.49, #queue-req: 483 2025-07-20 15:48:52,717 - __main__ - INFO - sglang running req: 10 queue req: 483 2025-07-20 15:48:53,613 - sglang - INFO - [2025-07-20 15:48:53 TP0] Prefill batch. #new-seq: 1, #new-token: 2289, #cached-token: 0, cache hit rate: 0.00%, token usage: 0.79, #running-req: 9, #queue-req: 482 2025-07-20 15:48:53,613 - __main__ - INFO - sglang running req: 9 queue req: 482 2025-07-20 15:48:54,440 - sglang - INFO - [2025-07-20 15:48:54 TP0] Decode batch. #running-req: 10, #token: 32272, token usage: 0.85, gen throughput (token/s): 231.56, #queue-req: 482 2025-07-20 15:48:54,440 - __main__ - INFO - sglang running req: 10 queue req: 482 2025-07-20 15:48:54,760 - __main__ - INFO - Queue remaining: 0 2025-07-20 15:48:54,760 - __main__ - INFO - Metric Name Lifetime (tokens/sec) Recently (tokens/sec) ---------------------------------------------------------------------------------- finished_input_tokens 28.71 137.88 finished_output_tokens 11.01 52.86 sglang_input_tokens 910.56 853.52 sglang_output_tokens 259.10 257.10 2025-07-20 15:48:54,760 - __main__ - INFO - Worker ID | errored | finished | started ----------+---------+----------+-------- 0 | 0 | 497 | 500 1 | 0 | 10 | 10 2 | 0 | 5 | 5 3 | 2 | 38 | 529 2025-07-20 15:48:55,431 - sglang - INFO - [2025-07-20 15:48:55 TP0] Decode batch. #running-req: 10, #token: 32672, token usage: 0.86, gen throughput (token/s): 403.38, #queue-req: 482 2025-07-20 15:48:55,432 - __main__ - INFO - sglang running req: 10 queue req: 482 2025-07-20 15:48:56,294 - sglang - INFO - [2025-07-20 15:48:56 TP0] Prefill batch. #new-seq: 1, #new-token: 2634, #cached-token: 0, cache hit rate: 0.00%, token usage: 0.73, #running-req: 8, #queue-req: 481 2025-07-20 15:48:56,294 - __main__ - INFO - sglang running req: 8 queue req: 481 2025-07-20 15:48:57,243 - sglang - INFO - [2025-07-20 15:48:57 TP0] Decode batch. #running-req: 9, #token: 30491, token usage: 0.80, gen throughput (token/s): 202.10, #queue-req: 481 2025-07-20 15:48:57,243 - __main__ - INFO - sglang running req: 9 queue req: 481 2025-07-20 15:48:57,586 - sglang - INFO - [2025-07-20 15:48:57 TP0] Prefill batch. #new-seq: 1, #new-token: 1150, #cached-token: 0, cache hit rate: 0.00%, token usage: 0.70, #running-req: 8, #queue-req: 480 2025-07-20 15:48:57,586 - __main__ - INFO - sglang running req: 8 queue req: 480 2025-07-20 15:48:58,728 - sglang - INFO - [2025-07-20 15:48:58 TP0] Decode batch. #running-req: 9, #token: 27948, token usage: 0.74, gen throughput (token/s): 241.60, #queue-req: 480 2025-07-20 15:48:58,729 - __main__ - INFO - sglang running req: 9 queue req: 480 2025-07-20 15:48:59,092 - sglang - INFO - [2025-07-20 15:48:59 TP0] Prefill batch. #new-seq: 1, #new-token: 2815, #cached-token: 0, cache hit rate: 0.00%, token usage: 0.71, #running-req: 8, #queue-req: 479 2025-07-20 15:48:59,092 - __main__ - INFO - sglang running req: 8 queue req: 479 2025-07-20 15:49:00,551 - sglang - INFO - [2025-07-20 15:49:00 TP0] Decode batch. #running-req: 9, #token: 29932, token usage: 0.79, gen throughput (token/s): 196.98, #queue-req: 479 2025-07-20 15:49:00,551 - __main__ - INFO - sglang running req: 9 queue req: 479 2025-07-20 15:49:01,536 - sglang - INFO - [2025-07-20 15:49:01 TP0] Decode batch. #running-req: 9, #token: 30292, token usage: 0.80, gen throughput (token/s): 365.35, #queue-req: 479 2025-07-20 15:49:01,537 - __main__ - INFO - sglang running req: 9 queue req: 479 2025-07-20 15:49:02,321 - sglang - INFO - [2025-07-20 15:49:02 TP0] Prefill batch. #new-seq: 1, #new-token: 4293, #cached-token: 0, cache hit rate: 0.00%, token usage: 0.72, #running-req: 8, #queue-req: 478 2025-07-20 15:49:02,322 - __main__ - INFO - sglang running req: 8 queue req: 478 2025-07-20 15:49:03,682 - sglang - INFO - [2025-07-20 15:49:03 TP0] Decode batch. #running-req: 9, #token: 31731, token usage: 0.84, gen throughput (token/s): 167.28, #queue-req: 478 2025-07-20 15:49:03,683 - __main__ - INFO - sglang running req: 9 queue req: 478 2025-07-20 15:49:04,672 - sglang - INFO - [2025-07-20 15:49:04 TP0] Decode batch. #running-req: 9, #token: 32091, token usage: 0.84, gen throughput (token/s): 363.89, #queue-req: 478 2025-07-20 15:49:04,672 - __main__ - INFO - sglang running req: 9 queue req: 478 2025-07-20 15:49:04,761 - __main__ - INFO - Queue remaining: 0 2025-07-20 15:49:04,762 - __main__ - INFO - Metric Name Lifetime (tokens/sec) Recently (tokens/sec) ---------------------------------------------------------------------------------- finished_input_tokens 28.51 137.88 finished_output_tokens 10.93 52.86 sglang_input_tokens 911.97 854.97 sglang_output_tokens 259.03 254.68 2025-07-20 15:49:04,762 - __main__ - INFO - Worker ID | errored | finished | started ----------+---------+----------+-------- 0 | 0 | 497 | 500 1 | 0 | 10 | 10 2 | 0 | 5 | 5 3 | 2 | 43 | 529 2025-07-20 15:49:05,660 - sglang - INFO - [2025-07-20 15:49:05 TP0] Decode batch. #running-req: 9, #token: 29351, token usage: 0.77, gen throughput (token/s): 364.33, #queue-req: 478 2025-07-20 15:49:05,660 - __main__ - INFO - sglang running req: 9 queue req: 478 2025-07-20 15:49:06,590 - sglang - INFO - [2025-07-20 15:49:06 TP0] Decode batch. #running-req: 8, #token: 29671, token usage: 0.78, gen throughput (token/s): 343.76, #queue-req: 478 2025-07-20 15:49:06,591 - __main__ - INFO - sglang running req: 8 queue req: 478 2025-07-20 15:49:06,777 - sglang - INFO - [2025-07-20 15:49:06 TP0] Prefill batch. #new-seq: 1, #new-token: 6873, #cached-token: 0, cache hit rate: 0.00%, token usage: 0.67, #running-req: 7, #queue-req: 477 2025-07-20 15:49:06,778 - __main__ - INFO - sglang running req: 7 queue req: 477 2025-07-20 15:49:08,736 - sglang - INFO - [2025-07-20 15:49:08 TP0] Prefill batch. #new-seq: 1, #new-token: 4350, #cached-token: 0, cache hit rate: 0.00%, token usage: 0.74, #running-req: 7, #queue-req: 476 2025-07-20 15:49:08,736 - __main__ - INFO - sglang running req: 7 queue req: 476 2025-07-20 15:49:10,105 - sglang - INFO - [2025-07-20 15:49:10 TP0] Prefill batch. #new-seq: 1, #new-token: 2543, #cached-token: 0, cache hit rate: 0.00%, token usage: 0.73, #running-req: 7, #queue-req: 475 2025-07-20 15:49:10,105 - __main__ - INFO - sglang running req: 7 queue req: 475 2025-07-20 15:49:11,179 - sglang - INFO - [2025-07-20 15:49:11 TP0] Decode batch. #running-req: 8, #token: 30209, token usage: 0.80, gen throughput (token/s): 69.08, #queue-req: 475 2025-07-20 15:49:11,180 - __main__ - INFO - sglang running req: 8 queue req: 475 2025-07-20 15:49:12,045 - sglang - INFO - [2025-07-20 15:49:12 TP0] Prefill batch. #new-seq: 1, #new-token: 2926, #cached-token: 0, cache hit rate: 0.00%, token usage: 0.72, #running-req: 7, #queue-req: 474 2025-07-20 15:49:12,045 - __main__ - INFO - sglang running req: 7 queue req: 474 2025-07-20 15:49:12,987 - sglang - INFO - [2025-07-20 15:49:12 TP0] Decode batch. #running-req: 8, #token: 30233, token usage: 0.80, gen throughput (token/s): 176.49, #queue-req: 474 2025-07-20 15:49:12,987 - __main__ - INFO - sglang running req: 8 queue req: 474 2025-07-20 15:49:13,920 - sglang - INFO - [2025-07-20 15:49:13 TP0] Decode batch. #running-req: 8, #token: 30553, token usage: 0.80, gen throughput (token/s): 342.84, #queue-req: 474 2025-07-20 15:49:13,920 - __main__ - INFO - sglang running req: 8 queue req: 474 2025-07-20 15:49:14,763 - __main__ - INFO - Queue remaining: 0 2025-07-20 15:49:14,764 - __main__ - INFO - Metric Name Lifetime (tokens/sec) Recently (tokens/sec) ---------------------------------------------------------------------------------- finished_input_tokens 28.31 137.88 finished_output_tokens 10.85 52.86 sglang_input_tokens 916.69 884.77 sglang_output_tokens 259.89 262.48 2025-07-20 15:49:14,764 - __main__ - INFO - Worker ID | errored | finished | started ----------+---------+----------+-------- 0 | 0 | 497 | 500 1 | 0 | 10 | 10 2 | 0 | 5 | 5 3 | 2 | 48 | 529 2025-07-20 15:49:14,854 - sglang - INFO - [2025-07-20 15:49:14 TP0] Decode batch. #running-req: 8, #token: 30873, token usage: 0.81, gen throughput (token/s): 342.70, #queue-req: 474 2025-07-20 15:49:14,854 - __main__ - INFO - sglang running req: 8 queue req: 474 2025-07-20 15:49:15,791 - sglang - INFO - [2025-07-20 15:49:15 TP0] Decode batch. #running-req: 8, #token: 31193, token usage: 0.82, gen throughput (token/s): 341.60, #queue-req: 474 2025-07-20 15:49:15,791 - __main__ - INFO - sglang running req: 8 queue req: 474 2025-07-20 15:49:16,400 - sglang - INFO - [2025-07-20 15:49:16 TP0] Prefill batch. #new-seq: 1, #new-token: 2657, #cached-token: 0, cache hit rate: 0.00%, token usage: 0.74, #running-req: 7, #queue-req: 473 2025-07-20 15:49:16,400 - __main__ - INFO - sglang running req: 7 queue req: 473 2025-07-20 15:49:17,552 - sglang - INFO - [2025-07-20 15:49:17 TP0] Decode batch. #running-req: 8, #token: 30904, token usage: 0.81, gen throughput (token/s): 181.06, #queue-req: 473 2025-07-20 15:49:17,553 - __main__ - INFO - sglang running req: 8 queue req: 473 2025-07-20 15:49:18,184 - sglang - INFO - [2025-07-20 15:49:18 TP0] Prefill batch. #new-seq: 1, #new-token: 2659, #cached-token: 0, cache hit rate: 0.00%, token usage: 0.74, #running-req: 7, #queue-req: 472 2025-07-20 15:49:18,184 - __main__ - INFO - sglang running req: 7 queue req: 472 2025-07-20 15:49:19,113 - __main__ - INFO - Process page scripts/data/11445200MB2D6222364440125017008.pdf-13 cancelled 2025-07-20 15:49:19,114 - __main__ - INFO - Process page scripts/data/11445224007035644H44421110A0001.pdf-3 cancelled 2025-07-20 15:49:19,114 - __main__ - INFO - Process page scripts/data/11445200MB2C47380T4440125017008 (1).pdf-12 cancelled 2025-07-20 15:49:19,114 - __main__ - INFO - Process page tests/gnarly_pdfs/guidebook_failed_pages.pdf-3 cancelled 2025-07-20 15:49:19,114 - __main__ - INFO - Process page tests/gnarly_pdfs/ti89_guidebook_programming.pdf-3 cancelled 2025-07-20 15:49:19,114 - __main__ - INFO - Process page tests/gnarly_pdfs/ti89_guidebook_programming.pdf-17 cancelled 2025-07-20 15:49:19,114 - __main__ - INFO - Process page tests/gnarly_pdfs/ti89_guidebook_programming.pdf-1 cancelled 2025-07-20 15:49:19,114 - __main__ - INFO - Process page tests/gnarly_pdfs/ti89_guidebook_programming.pdf-27 cancelled 2025-07-20 15:49:19,114 - __main__ - INFO - Process page tests/gnarly_pdfs/ti89_guidebook_programming.pdf-15 cancelled 2025-07-20 15:49:19,114 - __main__ - INFO - Process page tests/gnarly_pdfs/ti89_guidebook_programming.pdf-33 cancelled 2025-07-20 15:49:19,114 - __main__ - INFO - Process page tests/gnarly_pdfs/ti89_guidebook_programming.pdf-13 cancelled 2025-07-20 15:49:19,114 - __main__ - INFO - Process page tests/gnarly_pdfs/ti89_guidebook_programming.pdf-4 cancelled 2025-07-20 15:49:19,114 - __main__ - INFO - Process page tests/gnarly_pdfs/ti89_guidebook_programming.pdf-23 cancelled 2025-07-20 15:49:19,114 - __main__ - INFO - Process page tests/gnarly_pdfs/ti89_guidebook_programming.pdf-40 cancelled 2025-07-20 15:49:19,114 - __main__ - INFO - Process page tests/gnarly_pdfs/ti89_guidebook_programming.pdf-34 cancelled 2025-07-20 15:49:19,114 - __main__ - INFO - Process page tests/gnarly_pdfs/ti89_guidebook_programming.pdf-8 cancelled 2025-07-20 15:49:19,114 - __main__ - INFO - Process page tests/gnarly_pdfs/ti89_guidebook_programming.pdf-20 cancelled 2025-07-20 15:49:19,114 - __main__ - INFO - Process page tests/gnarly_pdfs/ti89_guidebook_programming.pdf-35 cancelled 2025-07-20 15:49:19,114 - __main__ - INFO - Process page tests/gnarly_pdfs/ti89_guidebook_programming.pdf-7 cancelled 2025-07-20 15:49:19,114 - __main__ - INFO - Process page tests/gnarly_pdfs/ti89_guidebook_programming.pdf-25 cancelled 2025-07-20 15:49:19,114 - __main__ - INFO - Process page tests/gnarly_pdfs/ti89_guidebook_programming.pdf-2 cancelled 2025-07-20 15:49:19,114 - __main__ - INFO - Process page tests/gnarly_pdfs/ti89_guidebook_programming.pdf-31 cancelled 2025-07-20 15:49:19,114 - __main__ - INFO - Process page tests/gnarly_pdfs/ti89_guidebook_programming.pdf-22 cancelled 2025-07-20 15:49:19,114 - __main__ - INFO - Process page tests/gnarly_pdfs/ti89_guidebook_programming.pdf-36 cancelled 2025-07-20 15:49:19,115 - __main__ - INFO - Process page tests/gnarly_pdfs/ti89_guidebook_programming.pdf-21 cancelled 2025-07-20 15:49:19,115 - __main__ - INFO - Process page tests/gnarly_pdfs/ti89_guidebook_programming.pdf-37 cancelled 2025-07-20 15:49:19,115 - __main__ - INFO - Process page tests/gnarly_pdfs/ti89_guidebook_programming.pdf-14 cancelled 2025-07-20 15:49:19,115 - __main__ - INFO - Process page tests/gnarly_pdfs/ti89_guidebook_programming.pdf-39 cancelled 2025-07-20 15:49:19,115 - __main__ - INFO - Process page tests/gnarly_pdfs/ti89_guidebook_programming.pdf-19 cancelled 2025-07-20 15:49:19,115 - __main__ - INFO - Process page tests/gnarly_pdfs/ti89_guidebook_programming.pdf-32 cancelled 2025-07-20 15:49:19,115 - __main__ - INFO - Process page tests/gnarly_pdfs/ti89_guidebook_programming.pdf-10 cancelled 2025-07-20 15:49:19,115 - __main__ - INFO - Process page tests/gnarly_pdfs/ti89_guidebook_programming.pdf-18 cancelled 2025-07-20 15:49:19,115 - __main__ - INFO - Process page tests/gnarly_pdfs/ti89_guidebook_programming.pdf-28 cancelled 2025-07-20 15:49:19,115 - __main__ - INFO - Process page tests/gnarly_pdfs/ti89_guidebook_programming.pdf-11 cancelled 2025-07-20 15:49:19,115 - __main__ - INFO - Process page tests/gnarly_pdfs/ti89_guidebook_programming.pdf-24 cancelled 2025-07-20 15:49:19,115 - __main__ - INFO - Process page tests/gnarly_pdfs/ti89_guidebook_programming.pdf-26 cancelled 2025-07-20 15:49:19,115 - __main__ - INFO - Process page tests/gnarly_pdfs/ti89_guidebook_programming.pdf-29 cancelled 2025-07-20 15:49:19,115 - __main__ - INFO - Process page tests/gnarly_pdfs/ti89_guidebook_programming.pdf-6 cancelled 2025-07-20 15:49:19,115 - __main__ - INFO - Process page tests/gnarly_pdfs/ti89_guidebook_programming.pdf-16 cancelled 2025-07-20 15:49:19,115 - __main__ - INFO - Process page tests/gnarly_pdfs/ti89_guidebook_programming.pdf-9 cancelled 2025-07-20 15:49:19,115 - __main__ - INFO - Process page tests/gnarly_pdfs/ti89_guidebook_programming.pdf-30 cancelled 2025-07-20 15:49:19,115 - __main__ - INFO - Process page tests/gnarly_pdfs/ti89_guidebook_programming.pdf-5 cancelled 2025-07-20 15:49:19,115 - __main__ - INFO - Process page tests/gnarly_pdfs/ti89_guidebook_programming.pdf-38 cancelled 2025-07-20 15:49:19,115 - __main__ - INFO - Process page tests/gnarly_pdfs/ti89_guidebook_programming.pdf-12 cancelled 2025-07-20 15:49:19,115 - __main__ - INFO - Process page tests/gnarly_pdfs/large_prompt_hint1.pdf-11 cancelled 2025-07-20 15:49:19,115 - __main__ - INFO - Process page tests/gnarly_pdfs/large_prompt_hint1.pdf-25 cancelled 2025-07-20 15:49:19,115 - __main__ - INFO - Process page tests/gnarly_pdfs/large_prompt_hint1.pdf-19 cancelled 2025-07-20 15:49:19,115 - __main__ - INFO - Process page tests/gnarly_pdfs/large_prompt_hint1.pdf-10 cancelled 2025-07-20 15:49:19,115 - __main__ - INFO - Process page tests/gnarly_pdfs/large_prompt_hint1.pdf-26 cancelled 2025-07-20 15:49:19,116 - __main__ - INFO - Process page tests/gnarly_pdfs/large_prompt_hint1.pdf-24 cancelled 2025-07-20 15:49:19,116 - __main__ - INFO - Process page tests/gnarly_pdfs/large_prompt_hint1.pdf-18 cancelled 2025-07-20 15:49:19,116 - __main__ - INFO - Process page tests/gnarly_pdfs/large_prompt_hint1.pdf-4 cancelled 2025-07-20 15:49:19,116 - __main__ - INFO - Process page tests/gnarly_pdfs/large_prompt_hint1.pdf-12 cancelled 2025-07-20 15:49:19,116 - __main__ - INFO - Process page tests/gnarly_pdfs/large_prompt_hint1.pdf-14 cancelled 2025-07-20 15:49:19,116 - __main__ - INFO - Process page tests/gnarly_pdfs/large_prompt_hint1.pdf-2 cancelled 2025-07-20 15:49:19,116 - __main__ - INFO - Process page tests/gnarly_pdfs/large_prompt_hint1.pdf-27 cancelled 2025-07-20 15:49:19,116 - __main__ - INFO - Process page tests/gnarly_pdfs/large_prompt_hint1.pdf-16 cancelled 2025-07-20 15:49:19,116 - __main__ - INFO - Process page tests/gnarly_pdfs/large_prompt_hint1.pdf-5 cancelled 2025-07-20 15:49:19,116 - __main__ - INFO - Process page tests/gnarly_pdfs/large_prompt_hint1.pdf-3 cancelled 2025-07-20 15:49:19,116 - __main__ - INFO - Process page tests/gnarly_pdfs/large_prompt_hint1.pdf-23 cancelled 2025-07-20 15:49:19,116 - __main__ - INFO - Process page tests/gnarly_pdfs/large_prompt_hint1.pdf-13 cancelled 2025-07-20 15:49:19,116 - __main__ - INFO - Process page tests/gnarly_pdfs/large_prompt_hint1.pdf-7 cancelled 2025-07-20 15:49:19,116 - __main__ - INFO - Process page tests/gnarly_pdfs/large_prompt_hint1.pdf-22 cancelled 2025-07-20 15:49:19,116 - __main__ - INFO - Process page tests/gnarly_pdfs/large_prompt_hint1.pdf-6 cancelled 2025-07-20 15:49:19,116 - __main__ - INFO - Process page tests/gnarly_pdfs/large_prompt_hint1.pdf-1 cancelled 2025-07-20 15:49:19,116 - __main__ - INFO - Process page tests/gnarly_pdfs/large_prompt_hint1.pdf-8 cancelled 2025-07-20 15:49:19,116 - __main__ - INFO - Process page tests/gnarly_pdfs/large_prompt_hint1.pdf-15 cancelled 2025-07-20 15:49:19,116 - __main__ - INFO - Process page tests/gnarly_pdfs/large_prompt_hint1.pdf-21 cancelled 2025-07-20 15:49:19,116 - __main__ - INFO - Process page tests/gnarly_pdfs/large_prompt_hint1.pdf-9 cancelled 2025-07-20 15:49:19,116 - __main__ - INFO - Process page tests/gnarly_pdfs/large_prompt_hint1.pdf-20 cancelled 2025-07-20 15:49:19,117 - __main__ - INFO - Process page tests/gnarly_pdfs/large_prompt_hint1.pdf-17 cancelled 2025-07-20 15:49:19,117 - __main__ - INFO - Process page tests/gnarly_pdfs/bws_book_ch2.pdf-46 cancelled 2025-07-20 15:49:19,117 - __main__ - INFO - Process page tests/gnarly_pdfs/bws_book_ch2.pdf-47 cancelled 2025-07-20 15:49:19,117 - __main__ - INFO - Process page tests/gnarly_pdfs/bws_book_ch2.pdf-33 cancelled 2025-07-20 15:49:19,117 - __main__ - INFO - Process page tests/gnarly_pdfs/bws_book_ch2.pdf-48 cancelled 2025-07-20 15:49:19,117 - __main__ - INFO - Process page tests/gnarly_pdfs/bws_book_ch2.pdf-38 cancelled 2025-07-20 15:49:19,117 - __main__ - INFO - Process page tests/gnarly_pdfs/bws_book_ch2.pdf-39 cancelled 2025-07-20 15:49:19,117 - __main__ - INFO - Process page tests/gnarly_pdfs/bws_book_ch2.pdf-12 cancelled 2025-07-20 15:49:19,117 - __main__ - INFO - Process page tests/gnarly_pdfs/bws_book_ch2.pdf-42 cancelled 2025-07-20 15:49:19,117 - __main__ - INFO - Process page tests/gnarly_pdfs/bws_book_ch2.pdf-26 cancelled 2025-07-20 15:49:19,117 - __main__ - INFO - Process page tests/gnarly_pdfs/bws_book_ch2.pdf-43 cancelled 2025-07-20 15:49:19,117 - __main__ - INFO - Process page tests/gnarly_pdfs/bws_book_ch2.pdf-44 cancelled 2025-07-20 15:49:19,117 - __main__ - INFO - Process page tests/gnarly_pdfs/bws_book_ch2.pdf-45 cancelled 2025-07-20 15:49:19,117 - __main__ - INFO - Process page tests/gnarly_pdfs/handwriting_bad_ocr.pdf-2 cancelled 2025-07-20 15:49:19,117 - __main__ - INFO - Process page tests/gnarly_pdfs/handwriting_bad_ocr.pdf-1 cancelled 2025-07-20 15:49:19,117 - __main__ - INFO - Process page tests/gnarly_pdfs/large_prompt_hint2.pdf-5 cancelled 2025-07-20 15:49:19,117 - __main__ - INFO - Process page tests/gnarly_pdfs/large_prompt_hint2.pdf-2 cancelled 2025-07-20 15:49:19,117 - __main__ - INFO - Process page tests/gnarly_pdfs/large_prompt_hint2.pdf-1 cancelled 2025-07-20 15:49:19,117 - __main__ - INFO - Process page tests/gnarly_pdfs/large_prompt_hint2.pdf-4 cancelled 2025-07-20 15:49:19,117 - __main__ - INFO - Process page tests/gnarly_pdfs/large_prompt_hint2.pdf-3 cancelled 2025-07-20 15:49:19,118 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_good_some_pages_should_get_filtered.pdf-12 cancelled 2025-07-20 15:49:19,118 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_good_some_pages_should_get_filtered.pdf-23 cancelled 2025-07-20 15:49:19,118 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_good_some_pages_should_get_filtered.pdf-13 cancelled 2025-07-20 15:49:19,118 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_good_some_pages_should_get_filtered.pdf-24 cancelled 2025-07-20 15:49:19,118 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_good_some_pages_should_get_filtered.pdf-14 cancelled 2025-07-20 15:49:19,118 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_good_some_pages_should_get_filtered.pdf-25 cancelled 2025-07-20 15:49:19,118 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_good_some_pages_should_get_filtered.pdf-3 cancelled 2025-07-20 15:49:19,118 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_good_some_pages_should_get_filtered.pdf-1 cancelled 2025-07-20 15:49:19,118 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_good_some_pages_should_get_filtered.pdf-15 cancelled 2025-07-20 15:49:19,118 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_good_some_pages_should_get_filtered.pdf-17 cancelled 2025-07-20 15:49:19,118 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_good_some_pages_should_get_filtered.pdf-8 cancelled 2025-07-20 15:49:19,118 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_good_some_pages_should_get_filtered.pdf-26 cancelled 2025-07-20 15:49:19,118 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_good_some_pages_should_get_filtered.pdf-2 cancelled 2025-07-20 15:49:19,118 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_good_some_pages_should_get_filtered.pdf-16 cancelled 2025-07-20 15:49:19,118 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_good_some_pages_should_get_filtered.pdf-18 cancelled 2025-07-20 15:49:19,118 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_good_some_pages_should_get_filtered.pdf-19 cancelled 2025-07-20 15:49:19,118 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_good_some_pages_should_get_filtered.pdf-5 cancelled 2025-07-20 15:49:19,118 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_good_some_pages_should_get_filtered.pdf-22 cancelled 2025-07-20 15:49:19,118 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_good_some_pages_should_get_filtered.pdf-9 cancelled 2025-07-20 15:49:19,118 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_good_some_pages_should_get_filtered.pdf-20 cancelled 2025-07-20 15:49:19,118 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_good_some_pages_should_get_filtered.pdf-6 cancelled 2025-07-20 15:49:19,118 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_good_some_pages_should_get_filtered.pdf-10 cancelled 2025-07-20 15:49:19,118 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_good_some_pages_should_get_filtered.pdf-21 cancelled 2025-07-20 15:49:19,119 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_good_some_pages_should_get_filtered.pdf-7 cancelled 2025-07-20 15:49:19,119 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_good_some_pages_should_get_filtered.pdf-11 cancelled 2025-07-20 15:49:19,119 - __main__ - INFO - Process page tests/gnarly_pdfs/large_prompt_hint3.pdf-1 cancelled 2025-07-20 15:49:19,119 - __main__ - INFO - Process page tests/gnarly_pdfs/large_prompt_hint3.pdf-4 cancelled 2025-07-20 15:49:19,119 - __main__ - INFO - Process page tests/gnarly_pdfs/large_prompt_hint3.pdf-2 cancelled 2025-07-20 15:49:19,119 - __main__ - INFO - Process page tests/gnarly_pdfs/large_prompt_hint3.pdf-3 cancelled 2025-07-20 15:49:19,119 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_images.pdf-1 cancelled 2025-07-20 15:49:19,119 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_images.pdf-8 cancelled 2025-07-20 15:49:19,119 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_images.pdf-61 cancelled 2025-07-20 15:49:19,119 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_images.pdf-18 cancelled 2025-07-20 15:49:19,119 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_images.pdf-40 cancelled 2025-07-20 15:49:19,119 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_images.pdf-30 cancelled 2025-07-20 15:49:19,119 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_images.pdf-51 cancelled 2025-07-20 15:49:19,119 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_images.pdf-9 cancelled 2025-07-20 15:49:19,119 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_images.pdf-62 cancelled 2025-07-20 15:49:19,119 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_images.pdf-19 cancelled 2025-07-20 15:49:19,119 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_images.pdf-41 cancelled 2025-07-20 15:49:19,119 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_images.pdf-31 cancelled 2025-07-20 15:49:19,119 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_images.pdf-52 cancelled 2025-07-20 15:49:19,119 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_images.pdf-10 cancelled 2025-07-20 15:49:19,119 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_images.pdf-63 cancelled 2025-07-20 15:49:19,119 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_images.pdf-20 cancelled 2025-07-20 15:49:19,119 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_images.pdf-42 cancelled 2025-07-20 15:49:19,119 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_images.pdf-32 cancelled 2025-07-20 15:49:19,119 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_images.pdf-53 cancelled 2025-07-20 15:49:19,119 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_images.pdf-11 cancelled 2025-07-20 15:49:19,120 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_images.pdf-64 cancelled 2025-07-20 15:49:19,120 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_images.pdf-21 cancelled 2025-07-20 15:49:19,120 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_images.pdf-43 cancelled 2025-07-20 15:49:19,120 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_images.pdf-33 cancelled 2025-07-20 15:49:19,120 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_images.pdf-54 cancelled 2025-07-20 15:49:19,120 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_images.pdf-12 cancelled 2025-07-20 15:49:19,120 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_images.pdf-65 cancelled 2025-07-20 15:49:19,120 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_images.pdf-22 cancelled 2025-07-20 15:49:19,120 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_images.pdf-44 cancelled 2025-07-20 15:49:19,120 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_images.pdf-2 cancelled 2025-07-20 15:49:19,120 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_images.pdf-34 cancelled 2025-07-20 15:49:19,120 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_images.pdf-55 cancelled 2025-07-20 15:49:19,120 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_images.pdf-13 cancelled 2025-07-20 15:49:19,120 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_images.pdf-66 cancelled 2025-07-20 15:49:19,120 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_images.pdf-23 cancelled 2025-07-20 15:49:19,120 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_images.pdf-45 cancelled 2025-07-20 15:49:19,120 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_images.pdf-3 cancelled 2025-07-20 15:49:19,134 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_images.pdf-35 cancelled 2025-07-20 15:49:19,134 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_images.pdf-56 cancelled 2025-07-20 15:49:19,134 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_images.pdf-14 cancelled 2025-07-20 15:49:19,135 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_images.pdf-67 cancelled 2025-07-20 15:49:19,135 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_images.pdf-24 cancelled 2025-07-20 15:49:19,135 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_images.pdf-46 cancelled 2025-07-20 15:49:19,135 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_images.pdf-4 cancelled 2025-07-20 15:49:19,135 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_images.pdf-36 cancelled 2025-07-20 15:49:19,135 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_images.pdf-50 cancelled 2025-07-20 15:49:19,135 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_images.pdf-57 cancelled 2025-07-20 15:49:19,136 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_images.pdf-15 cancelled 2025-07-20 15:49:19,136 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_images.pdf-68 cancelled 2025-07-20 15:49:19,136 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_images.pdf-25 cancelled 2025-07-20 15:49:19,136 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_images.pdf-47 cancelled 2025-07-20 15:49:19,136 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_images.pdf-5 cancelled 2025-07-20 15:49:19,136 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_images.pdf-37 cancelled 2025-07-20 15:49:19,136 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_images.pdf-58 cancelled 2025-07-20 15:49:19,137 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_images.pdf-16 cancelled 2025-07-20 15:49:19,137 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_images.pdf-27 cancelled 2025-07-20 15:49:19,137 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_images.pdf-48 cancelled 2025-07-20 15:49:19,137 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_images.pdf-6 cancelled 2025-07-20 15:49:19,137 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_images.pdf-38 cancelled 2025-07-20 15:49:19,137 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_images.pdf-59 cancelled 2025-07-20 15:49:19,137 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_images.pdf-17 cancelled 2025-07-20 15:49:19,137 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_images.pdf-28 cancelled 2025-07-20 15:49:19,137 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_images.pdf-49 cancelled 2025-07-20 15:49:19,137 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_images.pdf-7 cancelled 2025-07-20 15:49:19,137 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_images.pdf-39 cancelled 2025-07-20 15:49:19,137 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_images.pdf-60 cancelled 2025-07-20 15:49:19,138 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_images.pdf-26 cancelled 2025-07-20 15:49:19,138 - __main__ - INFO - Process page tests/gnarly_pdfs/slideshow_mostly_images.pdf-29 cancelled 2025-07-20 15:49:19,138 - __main__ - INFO - Process page tests/gnarly_pdfs/not_parsing2.pdf-5 cancelled 2025-07-20 15:49:19,138 - __main__ - INFO - Process page tests/gnarly_pdfs/not_parsing2.pdf-8 cancelled 2025-07-20 15:49:19,138 - __main__ - INFO - Process page tests/gnarly_pdfs/not_parsing2.pdf-2 cancelled 2025-07-20 15:49:19,138 - __main__ - INFO - Process page tests/gnarly_pdfs/not_parsing2.pdf-6 cancelled 2025-07-20 15:49:19,138 - __main__ - INFO - Process page tests/gnarly_pdfs/not_parsing2.pdf-9 cancelled 2025-07-20 15:49:19,138 - __main__ - INFO - Process page tests/gnarly_pdfs/not_parsing2.pdf-4 cancelled 2025-07-20 15:49:19,138 - __main__ - INFO - Process page tests/gnarly_pdfs/not_parsing2.pdf-1 cancelled 2025-07-20 15:49:19,138 - __main__ - INFO - Process page tests/gnarly_pdfs/not_parsing2.pdf-7 cancelled 2025-07-20 15:49:19,138 - __main__ - INFO - Process page tests/gnarly_pdfs/not_parsing2.pdf-3 cancelled 2025-07-20 15:49:19,138 - __main__ - INFO - Process page tests/gnarly_pdfs/load_v_error.pdf-6 cancelled 2025-07-20 15:49:19,138 - __main__ - INFO - Process page tests/gnarly_pdfs/load_v_error.pdf-14 cancelled 2025-07-20 15:49:19,138 - __main__ - INFO - Process page tests/gnarly_pdfs/load_v_error.pdf-1 cancelled 2025-07-20 15:49:19,139 - __main__ - INFO - Process page tests/gnarly_pdfs/load_v_error.pdf-9 cancelled 2025-07-20 15:49:19,139 - __main__ - INFO - Process page tests/gnarly_pdfs/load_v_error.pdf-4 cancelled 2025-07-20 15:49:19,139 - __main__ - INFO - Process page tests/gnarly_pdfs/load_v_error.pdf-12 cancelled 2025-07-20 15:49:19,139 - __main__ - INFO - Process page tests/gnarly_pdfs/load_v_error.pdf-7 cancelled 2025-07-20 15:49:19,139 - __main__ - INFO - Process page tests/gnarly_pdfs/load_v_error.pdf-15 cancelled 2025-07-20 15:49:19,139 - __main__ - INFO - Process page tests/gnarly_pdfs/load_v_error.pdf-16 cancelled 2025-07-20 15:49:19,139 - __main__ - INFO - Process page tests/gnarly_pdfs/load_v_error.pdf-2 cancelled 2025-07-20 15:49:19,139 - __main__ - INFO - Process page tests/gnarly_pdfs/load_v_error.pdf-10 cancelled 2025-07-20 15:49:19,139 - __main__ - INFO - Process page tests/gnarly_pdfs/load_v_error.pdf-5 cancelled 2025-07-20 15:49:19,139 - __main__ - INFO - Process page tests/gnarly_pdfs/load_v_error.pdf-13 cancelled 2025-07-20 15:49:19,139 - __main__ - INFO - Process page tests/gnarly_pdfs/load_v_error.pdf-8 cancelled 2025-07-20 15:49:19,139 - __main__ - INFO - Process page tests/gnarly_pdfs/load_v_error.pdf-3 cancelled 2025-07-20 15:49:19,139 - __main__ - INFO - Process page tests/gnarly_pdfs/load_v_error.pdf-11 cancelled 2025-07-20 15:49:19,139 - __main__ - INFO - Process page tests/gnarly_pdfs/small_page_size.pdf-1 cancelled 2025-07-20 15:49:19,139 - __main__ - INFO - Process page tests/gnarly_pdfs/discoverworld_crazy_tables.pdf-3 cancelled 2025-07-20 15:49:19,140 - __main__ - INFO - Process page tests/gnarly_pdfs/discoverworld_crazy_tables.pdf-7 cancelled 2025-07-20 15:49:19,140 - __main__ - INFO - Process page tests/gnarly_pdfs/discoverworld_crazy_tables.pdf-14 cancelled 2025-07-20 15:49:19,140 - __main__ - INFO - Process page tests/gnarly_pdfs/discoverworld_crazy_tables.pdf-25 cancelled 2025-07-20 15:49:19,140 - __main__ - INFO - Process page tests/gnarly_pdfs/discoverworld_crazy_tables.pdf-1 cancelled 2025-07-20 15:49:19,140 - __main__ - INFO - Process page tests/gnarly_pdfs/discoverworld_crazy_tables.pdf-11 cancelled 2025-07-20 15:49:19,140 - __main__ - INFO - Process page tests/gnarly_pdfs/discoverworld_crazy_tables.pdf-27 cancelled 2025-07-20 15:49:19,140 - __main__ - INFO - Process page tests/gnarly_pdfs/discoverworld_crazy_tables.pdf-10 cancelled 2025-07-20 15:49:19,140 - __main__ - INFO - Process page tests/gnarly_pdfs/discoverworld_crazy_tables.pdf-28 cancelled 2025-07-20 15:49:19,140 - __main__ - INFO - Process page tests/gnarly_pdfs/discoverworld_crazy_tables.pdf-5 cancelled 2025-07-20 15:49:19,140 - __main__ - INFO - Process page tests/gnarly_pdfs/discoverworld_crazy_tables.pdf-8 cancelled 2025-07-20 15:49:19,140 - __main__ - INFO - Process page tests/gnarly_pdfs/discoverworld_crazy_tables.pdf-29 cancelled 2025-07-20 15:49:19,140 - __main__ - INFO - Process page tests/gnarly_pdfs/discoverworld_crazy_tables.pdf-12 cancelled 2025-07-20 15:49:19,140 - __main__ - INFO - Process page tests/gnarly_pdfs/discoverworld_crazy_tables.pdf-13 cancelled 2025-07-20 15:49:19,140 - __main__ - INFO - Process page tests/gnarly_pdfs/discoverworld_crazy_tables.pdf-9 cancelled 2025-07-20 15:49:19,141 - __main__ - INFO - Process page tests/gnarly_pdfs/discoverworld_crazy_tables.pdf-15 cancelled 2025-07-20 15:49:19,141 - __main__ - INFO - Process page tests/gnarly_pdfs/discoverworld_crazy_tables.pdf-17 cancelled 2025-07-20 15:49:19,141 - __main__ - INFO - Process page tests/gnarly_pdfs/discoverworld_crazy_tables.pdf-19 cancelled 2025-07-20 15:49:19,141 - __main__ - INFO - Process page tests/gnarly_pdfs/discoverworld_crazy_tables.pdf-4 cancelled 2025-07-20 15:49:19,141 - __main__ - INFO - Process page tests/gnarly_pdfs/discoverworld_crazy_tables.pdf-26 cancelled 2025-07-20 15:49:19,141 - __main__ - INFO - Process page tests/gnarly_pdfs/discoverworld_crazy_tables.pdf-21 cancelled 2025-07-20 15:49:19,141 - __main__ - INFO - Process page tests/gnarly_pdfs/discoverworld_crazy_tables.pdf-2 cancelled 2025-07-20 15:49:19,141 - __main__ - INFO - Process page tests/gnarly_pdfs/discoverworld_crazy_tables.pdf-20 cancelled 2025-07-20 15:49:19,141 - __main__ - INFO - Process page tests/gnarly_pdfs/discoverworld_crazy_tables.pdf-22 cancelled 2025-07-20 15:49:19,141 - __main__ - INFO - Process page tests/gnarly_pdfs/discoverworld_crazy_tables.pdf-6 cancelled 2025-07-20 15:49:19,141 - __main__ - INFO - Process page tests/gnarly_pdfs/discoverworld_crazy_tables.pdf-16 cancelled 2025-07-20 15:49:19,141 - __main__ - INFO - Process page tests/gnarly_pdfs/discoverworld_crazy_tables.pdf-23 cancelled 2025-07-20 15:49:19,141 - __main__ - INFO - Process page tests/gnarly_pdfs/discoverworld_crazy_tables.pdf-18 cancelled 2025-07-20 15:49:19,141 - __main__ - INFO - Process page tests/gnarly_pdfs/discoverworld_crazy_tables.pdf-24 cancelled 2025-07-20 15:49:19,142 - __main__ - INFO - Process page tests/gnarly_pdfs/olmo-page-1.pdf-1 cancelled 2025-07-20 15:49:19,142 - __main__ - INFO - Process page tests/gnarly_pdfs/lots_of_chem_tables.pdf-2 cancelled 2025-07-20 15:49:19,142 - __main__ - INFO - Process page tests/gnarly_pdfs/lots_of_chem_tables.pdf-8 cancelled 2025-07-20 15:49:19,142 - __main__ - INFO - Process page tests/gnarly_pdfs/lots_of_chem_tables.pdf-4 cancelled 2025-07-20 15:49:19,142 - __main__ - INFO - Process page tests/gnarly_pdfs/lots_of_chem_tables.pdf-6 cancelled 2025-07-20 15:49:19,142 - __main__ - INFO - Process page tests/gnarly_pdfs/lots_of_chem_tables.pdf-3 cancelled 2025-07-20 15:49:19,142 - __main__ - INFO - Process page tests/gnarly_pdfs/lots_of_chem_tables.pdf-1 cancelled 2025-07-20 15:49:19,142 - __main__ - INFO - Process page tests/gnarly_pdfs/lots_of_chem_tables.pdf-9 cancelled 2025-07-20 15:49:19,142 - __main__ - INFO - Process page tests/gnarly_pdfs/lots_of_chem_tables.pdf-5 cancelled 2025-07-20 15:49:19,142 - __main__ - INFO - Process page tests/gnarly_pdfs/lots_of_chem_tables.pdf-7 cancelled 2025-07-20 15:49:19,142 - __main__ - INFO - Process page tests/gnarly_pdfs/some_ocr1.pdf-1 cancelled 2025-07-20 15:49:19,142 - __main__ - INFO - Process page tests/gnarly_pdfs/dolma-page-1.pdf-1 cancelled 2025-07-20 15:49:19,142 - __main__ - INFO - Process page tests/gnarly_pdfs/overrun_on_pg8.pdf-10 cancelled 2025-07-20 15:49:19,142 - __main__ - INFO - Process page tests/gnarly_pdfs/overrun_on_pg8.pdf-23 cancelled 2025-07-20 15:49:19,142 - __main__ - INFO - Process page tests/gnarly_pdfs/overrun_on_pg8.pdf-39 cancelled 2025-07-20 15:49:19,143 - __main__ - INFO - Process page tests/gnarly_pdfs/overrun_on_pg8.pdf-27 cancelled 2025-07-20 15:49:19,143 - __main__ - INFO - Process page tests/gnarly_pdfs/overrun_on_pg8.pdf-50 cancelled 2025-07-20 15:49:19,143 - __main__ - INFO - Process page tests/gnarly_pdfs/overrun_on_pg8.pdf-1 cancelled 2025-07-20 15:49:19,143 - __main__ - INFO - Process page tests/gnarly_pdfs/overrun_on_pg8.pdf-11 cancelled 2025-07-20 15:49:19,143 - __main__ - INFO - Process page tests/gnarly_pdfs/overrun_on_pg8.pdf-24 cancelled 2025-07-20 15:49:19,143 - __main__ - INFO - Process page tests/gnarly_pdfs/overrun_on_pg8.pdf-40 cancelled 2025-07-20 15:49:19,143 - __main__ - INFO - Process page tests/gnarly_pdfs/overrun_on_pg8.pdf-28 cancelled 2025-07-20 15:49:19,143 - __main__ - INFO - Process page tests/gnarly_pdfs/overrun_on_pg8.pdf-51 cancelled 2025-07-20 15:49:19,143 - __main__ - INFO - Process page tests/gnarly_pdfs/overrun_on_pg8.pdf-15 cancelled 2025-07-20 15:49:19,143 - __main__ - INFO - Process page tests/gnarly_pdfs/overrun_on_pg8.pdf-12 cancelled 2025-07-20 15:49:19,143 - __main__ - INFO - Process page tests/gnarly_pdfs/overrun_on_pg8.pdf-41 cancelled 2025-07-20 15:49:19,143 - __main__ - INFO - Process page tests/gnarly_pdfs/overrun_on_pg8.pdf-29 cancelled 2025-07-20 15:49:19,143 - __main__ - INFO - Process page tests/gnarly_pdfs/overrun_on_pg8.pdf-52 cancelled 2025-07-20 15:49:19,143 - __main__ - INFO - Process page tests/gnarly_pdfs/overrun_on_pg8.pdf-2 cancelled 2025-07-20 15:49:19,143 - __main__ - INFO - Process page tests/gnarly_pdfs/overrun_on_pg8.pdf-16 cancelled 2025-07-20 15:49:19,144 - __main__ - INFO - Process page tests/gnarly_pdfs/overrun_on_pg8.pdf-13 cancelled 2025-07-20 15:49:19,144 - __main__ - INFO - Process page tests/gnarly_pdfs/overrun_on_pg8.pdf-42 cancelled 2025-07-20 15:49:19,144 - __main__ - INFO - Process page tests/gnarly_pdfs/overrun_on_pg8.pdf-30 cancelled 2025-07-20 15:49:19,144 - __main__ - INFO - Process page tests/gnarly_pdfs/overrun_on_pg8.pdf-53 cancelled 2025-07-20 15:49:19,144 - __main__ - INFO - Process page tests/gnarly_pdfs/overrun_on_pg8.pdf-3 cancelled 2025-07-20 15:49:19,144 - __main__ - INFO - Process page tests/gnarly_pdfs/overrun_on_pg8.pdf-17 cancelled 2025-07-20 15:49:19,144 - __main__ - INFO - Process page tests/gnarly_pdfs/overrun_on_pg8.pdf-43 cancelled 2025-07-20 15:49:19,144 - __main__ - INFO - Process page tests/gnarly_pdfs/overrun_on_pg8.pdf-31 cancelled 2025-07-20 15:49:19,144 - __main__ - INFO - Process page tests/gnarly_pdfs/overrun_on_pg8.pdf-54 cancelled 2025-07-20 15:49:19,144 - __main__ - INFO - Process page tests/gnarly_pdfs/overrun_on_pg8.pdf-4 cancelled 2025-07-20 15:49:19,144 - __main__ - INFO - Process page tests/gnarly_pdfs/overrun_on_pg8.pdf-26 cancelled 2025-07-20 15:49:19,144 - __main__ - INFO - Process page tests/gnarly_pdfs/overrun_on_pg8.pdf-14 cancelled 2025-07-20 15:49:19,144 - __main__ - INFO - Process page tests/gnarly_pdfs/overrun_on_pg8.pdf-44 cancelled 2025-07-20 15:49:19,144 - __main__ - INFO - Process page tests/gnarly_pdfs/overrun_on_pg8.pdf-32 cancelled 2025-07-20 15:49:19,145 - __main__ - INFO - Process page tests/gnarly_pdfs/overrun_on_pg8.pdf-36 cancelled 2025-07-20 15:49:19,145 - __main__ - INFO - Process page tests/gnarly_pdfs/overrun_on_pg8.pdf-5 cancelled 2025-07-20 15:49:19,145 - __main__ - INFO - Process page tests/gnarly_pdfs/overrun_on_pg8.pdf-18 cancelled 2025-07-20 15:49:19,145 - __main__ - INFO - Process page tests/gnarly_pdfs/overrun_on_pg8.pdf-45 cancelled 2025-07-20 15:49:19,145 - __main__ - INFO - Process page tests/gnarly_pdfs/overrun_on_pg8.pdf-33 cancelled 2025-07-20 15:49:19,145 - __main__ - INFO - Process page tests/gnarly_pdfs/overrun_on_pg8.pdf-6 cancelled 2025-07-20 15:49:19,145 - __main__ - INFO - Process page tests/gnarly_pdfs/overrun_on_pg8.pdf-19 cancelled 2025-07-20 15:49:19,145 - __main__ - INFO - Process page tests/gnarly_pdfs/overrun_on_pg8.pdf-46 cancelled 2025-07-20 15:49:19,145 - __main__ - INFO - Process page tests/gnarly_pdfs/overrun_on_pg8.pdf-34 cancelled 2025-07-20 15:49:19,145 - __main__ - INFO - Process page tests/gnarly_pdfs/overrun_on_pg8.pdf-7 cancelled 2025-07-20 15:49:19,145 - __main__ - INFO - Process page tests/gnarly_pdfs/overrun_on_pg8.pdf-20 cancelled 2025-07-20 15:49:19,145 - __main__ - INFO - Process page tests/gnarly_pdfs/overrun_on_pg8.pdf-47 cancelled 2025-07-20 15:49:19,145 - __main__ - INFO - Process page tests/gnarly_pdfs/overrun_on_pg8.pdf-35 cancelled 2025-07-20 15:49:19,145 - __main__ - INFO - Process page tests/gnarly_pdfs/overrun_on_pg8.pdf-8 cancelled 2025-07-20 15:49:19,145 - __main__ - INFO - Process page tests/gnarly_pdfs/overrun_on_pg8.pdf-48 cancelled 2025-07-20 15:49:19,146 - __main__ - INFO - Process page tests/gnarly_pdfs/overrun_on_pg8.pdf-25 cancelled 2025-07-20 15:49:19,146 - __main__ - INFO - Process page tests/gnarly_pdfs/overrun_on_pg8.pdf-9 cancelled 2025-07-20 15:49:19,146 - __main__ - INFO - Process page tests/gnarly_pdfs/overrun_on_pg8.pdf-22 cancelled 2025-07-20 15:49:19,146 - __main__ - INFO - Process page tests/gnarly_pdfs/overrun_on_pg8.pdf-49 cancelled 2025-07-20 15:49:19,146 - __main__ - INFO - Process page tests/gnarly_pdfs/lots_of_sci_tables.pdf-4 cancelled 2025-07-20 15:49:19,146 - __main__ - INFO - Process page tests/gnarly_pdfs/lots_of_sci_tables.pdf-2 cancelled 2025-07-20 15:49:19,146 - __main__ - INFO - Process page tests/gnarly_pdfs/lots_of_sci_tables.pdf-5 cancelled 2025-07-20 15:49:19,146 - __main__ - INFO - Process page tests/gnarly_pdfs/lots_of_sci_tables.pdf-3 cancelled 2025-07-20 15:49:19,146 - __main__ - INFO - Process page tests/gnarly_pdfs/lots_of_sci_tables.pdf-6 cancelled 2025-07-20 15:49:19,146 - __main__ - INFO - Process page tests/gnarly_pdfs/lots_of_sci_tables.pdf-1 cancelled 2025-07-20 15:49:19,146 - __main__ - INFO - Process page tests/gnarly_pdfs/newspaper.pdf-1 cancelled 2025-07-20 15:49:19,146 - __main__ - INFO - Process page tests/gnarly_pdfs/pdftotext_two_column_issue.pdf-2 cancelled 2025-07-20 15:49:19,146 - __main__ - INFO - Process page tests/gnarly_pdfs/pdftotext_two_column_issue.pdf-10 cancelled 2025-07-20 15:49:19,146 - __main__ - INFO - Process page tests/gnarly_pdfs/pdftotext_two_column_issue.pdf-5 cancelled 2025-07-20 15:49:19,146 - __main__ - INFO - Process page tests/gnarly_pdfs/pdftotext_two_column_issue.pdf-13 cancelled 2025-07-20 15:49:19,146 - __main__ - INFO - Process page tests/gnarly_pdfs/pdftotext_two_column_issue.pdf-12 cancelled 2025-07-20 15:49:19,147 - __main__ - INFO - Process page tests/gnarly_pdfs/pdftotext_two_column_issue.pdf-8 cancelled 2025-07-20 15:49:19,147 - __main__ - INFO - Process page tests/gnarly_pdfs/pdftotext_two_column_issue.pdf-3 cancelled 2025-07-20 15:49:19,147 - __main__ - INFO - Process page tests/gnarly_pdfs/pdftotext_two_column_issue.pdf-11 cancelled 2025-07-20 15:49:19,147 - __main__ - INFO - Process page tests/gnarly_pdfs/pdftotext_two_column_issue.pdf-6 cancelled 2025-07-20 15:49:19,147 - __main__ - INFO - Process page tests/gnarly_pdfs/pdftotext_two_column_issue.pdf-14 cancelled 2025-07-20 15:49:19,147 - __main__ - INFO - Process page tests/gnarly_pdfs/pdftotext_two_column_issue.pdf-1 cancelled 2025-07-20 15:49:19,147 - __main__ - INFO - Process page tests/gnarly_pdfs/pdftotext_two_column_issue.pdf-9 cancelled 2025-07-20 15:49:19,147 - __main__ - INFO - Process page tests/gnarly_pdfs/pdftotext_two_column_issue.pdf-4 cancelled 2025-07-20 15:49:19,147 - __main__ - INFO - Process page tests/gnarly_pdfs/pdftotext_two_column_issue.pdf-7 cancelled 2025-07-20 15:49:19,147 - __main__ - INFO - Process page tests/gnarly_pdfs/repeating_references_on_pg9_pg10.pdf-4 cancelled 2025-07-20 15:49:19,147 - __main__ - INFO - Process page tests/gnarly_pdfs/repeating_references_on_pg9_pg10.pdf-10 cancelled 2025-07-20 15:49:19,147 - __main__ - INFO - Process page tests/gnarly_pdfs/repeating_references_on_pg9_pg10.pdf-6 cancelled 2025-07-20 15:49:19,147 - __main__ - INFO - Process page tests/gnarly_pdfs/repeating_references_on_pg9_pg10.pdf-2 cancelled 2025-07-20 15:49:19,147 - __main__ - INFO - Process page tests/gnarly_pdfs/repeating_references_on_pg9_pg10.pdf-8 cancelled 2025-07-20 15:49:19,147 - __main__ - INFO - Process page tests/gnarly_pdfs/repeating_references_on_pg9_pg10.pdf-9 cancelled 2025-07-20 15:49:19,148 - __main__ - INFO - Process page tests/gnarly_pdfs/repeating_references_on_pg9_pg10.pdf-7 cancelled 2025-07-20 15:49:19,148 - __main__ - INFO - Process page tests/gnarly_pdfs/repeating_references_on_pg9_pg10.pdf-3 cancelled 2025-07-20 15:49:19,148 - __main__ - INFO - Process page tests/gnarly_pdfs/repeating_references_on_pg9_pg10.pdf-5 cancelled 2025-07-20 15:49:19,148 - __main__ - INFO - Process page tests/gnarly_pdfs/repeating_references_on_pg9_pg10.pdf-1 cancelled 2025-07-20 15:49:19,148 - __main__ - INFO - Process page tests/gnarly_pdfs/skinnypage.pdf-1 cancelled 2025-07-20 15:49:19,148 - __main__ - INFO - Process page tests/gnarly_pdfs/not_parsing.pdf-6 cancelled 2025-07-20 15:49:19,148 - __main__ - INFO - Process page tests/gnarly_pdfs/not_parsing.pdf-1 cancelled 2025-07-20 15:49:19,148 - __main__ - INFO - Process page tests/gnarly_pdfs/not_parsing.pdf-4 cancelled 2025-07-20 15:49:19,148 - __main__ - INFO - Process page tests/gnarly_pdfs/not_parsing.pdf-7 cancelled 2025-07-20 15:49:19,148 - __main__ - INFO - Process page tests/gnarly_pdfs/not_parsing.pdf-2 cancelled 2025-07-20 15:49:19,148 - __main__ - INFO - Process page tests/gnarly_pdfs/not_parsing.pdf-5 cancelled 2025-07-20 15:49:19,148 - __main__ - INFO - Process page tests/gnarly_pdfs/not_parsing.pdf-8 cancelled 2025-07-20 15:49:19,148 - __main__ - INFO - Process page tests/gnarly_pdfs/not_parsing.pdf-3 cancelled 2025-07-20 15:49:19,148 - __main__ - INFO - Process page tests/gnarly_pdfs/most_content_in_image_form.pdf-3 cancelled 2025-07-20 15:49:19,148 - __main__ - INFO - Process page tests/gnarly_pdfs/most_content_in_image_form.pdf-6 cancelled 2025-07-20 15:49:19,148 - __main__ - INFO - Process page tests/gnarly_pdfs/most_content_in_image_form.pdf-1 cancelled 2025-07-20 15:49:19,149 - __main__ - INFO - Process page tests/gnarly_pdfs/most_content_in_image_form.pdf-4 cancelled 2025-07-20 15:49:19,149 - __main__ - INFO - Process page tests/gnarly_pdfs/most_content_in_image_form.pdf-7 cancelled 2025-07-20 15:49:19,149 - __main__ - INFO - Process page tests/gnarly_pdfs/most_content_in_image_form.pdf-2 cancelled 2025-07-20 15:49:19,149 - __main__ - INFO - Process page tests/gnarly_pdfs/most_content_in_image_form.pdf-5 cancelled 2025-07-20 15:49:19,149 - __main__ - INFO - Process page tests/gnarly_pdfs/failing_pdf_pg9.pdf-2 cancelled 2025-07-20 15:49:19,149 - __main__ - INFO - Process page tests/gnarly_pdfs/failing_pdf_pg9.pdf-4 cancelled 2025-07-20 15:49:19,149 - __main__ - INFO - Process page tests/gnarly_pdfs/failing_pdf_pg9.pdf-8 cancelled 2025-07-20 15:49:19,149 - __main__ - INFO - Process page tests/gnarly_pdfs/failing_pdf_pg9.pdf-3 cancelled 2025-07-20 15:49:19,149 - __main__ - INFO - Process page tests/gnarly_pdfs/failing_pdf_pg9.pdf-6 cancelled 2025-07-20 15:49:19,149 - __main__ - INFO - Process page tests/gnarly_pdfs/failing_pdf_pg9.pdf-5 cancelled 2025-07-20 15:49:19,149 - __main__ - INFO - Process page tests/gnarly_pdfs/failing_pdf_pg9.pdf-9 cancelled 2025-07-20 15:49:19,149 - __main__ - INFO - Process page tests/gnarly_pdfs/failing_pdf_pg9.pdf-1 cancelled 2025-07-20 15:49:19,149 - __main__ - INFO - Process page tests/gnarly_pdfs/failing_pdf_pg9.pdf-7 cancelled 2025-07-20 15:49:19,149 - __main__ - INFO - Process page tests/gnarly_pdfs/failing_anchor_pg4.pdf-8 cancelled 2025-07-20 15:49:19,149 - __main__ - INFO - Process page tests/gnarly_pdfs/failing_anchor_pg4.pdf-5 cancelled 2025-07-20 15:49:19,149 - __main__ - INFO - Process page tests/gnarly_pdfs/failing_anchor_pg4.pdf-3 cancelled 2025-07-20 15:49:19,150 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-77 cancelled 2025-07-20 15:49:19,150 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-43 cancelled 2025-07-20 15:49:19,150 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-68 cancelled 2025-07-20 15:49:19,150 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-78 cancelled 2025-07-20 15:49:19,150 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-26 cancelled 2025-07-20 15:49:19,150 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-61 cancelled 2025-07-20 15:49:19,150 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-76 cancelled 2025-07-20 15:49:19,150 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-36 cancelled 2025-07-20 15:49:19,150 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-59 cancelled 2025-07-20 15:49:19,150 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-80 cancelled 2025-07-20 15:49:19,150 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-16 cancelled 2025-07-20 15:49:19,150 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-64 cancelled 2025-07-20 15:49:19,150 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-81 cancelled 2025-07-20 15:49:19,150 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-46 cancelled 2025-07-20 15:49:19,150 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-71 cancelled 2025-07-20 15:49:19,150 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-82 cancelled 2025-07-20 15:49:19,151 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-21 cancelled 2025-07-20 15:49:19,151 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-65 cancelled 2025-07-20 15:49:19,151 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-83 cancelled 2025-07-20 15:49:19,151 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-33 cancelled 2025-07-20 15:49:19,151 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-63 cancelled 2025-07-20 15:49:19,151 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-84 cancelled 2025-07-20 15:49:19,151 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-30 cancelled 2025-07-20 15:49:19,151 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-69 cancelled 2025-07-20 15:49:19,151 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-79 cancelled 2025-07-20 15:49:19,151 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-31 cancelled 2025-07-20 15:49:19,151 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-67 cancelled 2025-07-20 15:49:19,151 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-73 cancelled 2025-07-20 15:49:19,151 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-15 cancelled 2025-07-20 15:49:19,151 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-74 cancelled 2025-07-20 15:49:19,151 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-57 cancelled 2025-07-20 15:49:19,151 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-70 cancelled 2025-07-20 15:49:19,151 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-75 cancelled 2025-07-20 15:49:19,152 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-19 cancelled 2025-07-20 15:49:19,152 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-60 cancelled 2025-07-20 15:49:19,152 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-72 cancelled 2025-07-20 15:49:19,152 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-18 cancelled 2025-07-20 15:49:19,152 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-53 cancelled 2025-07-20 15:49:19,152 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-55 cancelled 2025-07-20 15:49:19,152 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-23 cancelled 2025-07-20 15:49:19,152 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-45 cancelled 2025-07-20 15:49:19,152 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-27 cancelled 2025-07-20 15:49:19,152 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-52 cancelled 2025-07-20 15:49:19,152 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-94 cancelled 2025-07-20 15:49:19,152 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-14 cancelled 2025-07-20 15:49:19,152 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-66 cancelled 2025-07-20 15:49:19,152 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-28 cancelled 2025-07-20 15:49:19,152 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-9 cancelled 2025-07-20 15:49:19,152 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-58 cancelled 2025-07-20 15:49:19,153 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-90 cancelled 2025-07-20 15:49:19,153 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-20 cancelled 2025-07-20 15:49:19,153 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-56 cancelled 2025-07-20 15:49:19,153 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-17 cancelled 2025-07-20 15:49:19,153 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-51 cancelled 2025-07-20 15:49:19,153 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-89 cancelled 2025-07-20 15:49:19,153 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-44 cancelled 2025-07-20 15:49:19,153 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-86 cancelled 2025-07-20 15:49:19,153 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-25 cancelled 2025-07-20 15:49:19,153 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-62 cancelled 2025-07-20 15:49:19,153 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-39 cancelled 2025-07-20 15:49:19,153 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-87 cancelled 2025-07-20 15:49:19,153 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-12 cancelled 2025-07-20 15:49:19,153 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-40 cancelled 2025-07-20 15:49:19,153 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-88 cancelled 2025-07-20 15:49:19,153 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-22 cancelled 2025-07-20 15:49:19,153 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-96 cancelled 2025-07-20 15:49:19,154 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-54 cancelled 2025-07-20 15:49:19,154 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-6 cancelled 2025-07-20 15:49:19,154 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-38 cancelled 2025-07-20 15:49:19,154 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-2 cancelled 2025-07-20 15:49:19,154 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-105 cancelled 2025-07-20 15:49:19,154 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-37 cancelled 2025-07-20 15:49:19,154 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-5 cancelled 2025-07-20 15:49:19,154 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-106 cancelled 2025-07-20 15:49:19,154 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-35 cancelled 2025-07-20 15:49:19,154 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-10 cancelled 2025-07-20 15:49:19,154 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-102 cancelled 2025-07-20 15:49:19,154 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-42 cancelled 2025-07-20 15:49:19,154 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-99 cancelled 2025-07-20 15:49:19,154 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-34 cancelled 2025-07-20 15:49:19,154 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-103 cancelled 2025-07-20 15:49:19,154 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-47 cancelled 2025-07-20 15:49:19,154 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-104 cancelled 2025-07-20 15:49:19,154 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-50 cancelled 2025-07-20 15:49:19,155 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-85 cancelled 2025-07-20 15:49:19,155 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-7 cancelled 2025-07-20 15:49:19,155 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-101 cancelled 2025-07-20 15:49:19,155 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-24 cancelled 2025-07-20 15:49:19,155 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-4 cancelled 2025-07-20 15:49:19,155 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-92 cancelled 2025-07-20 15:49:19,155 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-48 cancelled 2025-07-20 15:49:19,155 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-11 cancelled 2025-07-20 15:49:19,155 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-98 cancelled 2025-07-20 15:49:19,155 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-41 cancelled 2025-07-20 15:49:19,155 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-97 cancelled 2025-07-20 15:49:19,155 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-29 cancelled 2025-07-20 15:49:19,155 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-1 cancelled 2025-07-20 15:49:19,155 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-91 cancelled 2025-07-20 15:49:19,155 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-3 cancelled 2025-07-20 15:49:19,155 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-100 cancelled 2025-07-20 15:49:19,155 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-32 cancelled 2025-07-20 15:49:19,156 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-93 cancelled 2025-07-20 15:49:19,156 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-49 cancelled 2025-07-20 15:49:19,156 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-8 cancelled 2025-07-20 15:49:19,156 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-95 cancelled 2025-07-20 15:49:19,156 - __main__ - INFO - Process page tests/gnarly_pdfs/instructions_and_schematics.pdf-13 cancelled 2025-07-20 15:49:19,156 - __main__ - INFO - Process page tests/gnarly_pdfs/form_on_later_pages.pdf-5 cancelled 2025-07-20 15:49:19,156 - __main__ - INFO - Process page tests/gnarly_pdfs/form_on_later_pages.pdf-8 cancelled 2025-07-20 15:49:19,156 - __main__ - INFO - Process page tests/gnarly_pdfs/form_on_later_pages.pdf-3 cancelled 2025-07-20 15:49:19,156 - __main__ - INFO - Process page tests/gnarly_pdfs/form_on_later_pages.pdf-10 cancelled 2025-07-20 15:49:19,156 - __main__ - INFO - Process page tests/gnarly_pdfs/form_on_later_pages.pdf-6 cancelled 2025-07-20 15:49:19,156 - __main__ - INFO - Process page tests/gnarly_pdfs/form_on_later_pages.pdf-1 cancelled 2025-07-20 15:49:19,156 - __main__ - INFO - Process page tests/gnarly_pdfs/form_on_later_pages.pdf-9 cancelled 2025-07-20 15:49:19,156 - __main__ - INFO - Process page tests/gnarly_pdfs/form_on_later_pages.pdf-4 cancelled 2025-07-20 15:49:19,156 - __main__ - INFO - Process page tests/gnarly_pdfs/form_on_later_pages.pdf-7 cancelled 2025-07-20 15:49:19,156 - __main__ - INFO - Process page tests/gnarly_pdfs/form_on_later_pages.pdf-2 cancelled 2025-07-20 15:49:19,157 - sglang - INFO - Process Process-2: 2025-07-20 15:49:19,157 - sglang - INFO - Process Process-1: 2025-07-20 15:49:19,157 - sglang - INFO - Traceback (most recent call last): 2025-07-20 15:49:19,157 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/multiprocessing/process.py", line 314, in _bootstrap 2025-07-20 15:49:19,157 - sglang - INFO - self.run() 2025-07-20 15:49:19,157 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/multiprocessing/process.py", line 108, in run 2025-07-20 15:49:19,157 - sglang - INFO - self._target(*self._args, **self._kwargs) 2025-07-20 15:49:19,158 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/managers/scheduler.py", line 1784, in run_scheduler_process 2025-07-20 15:49:19,158 - sglang - INFO - scheduler.event_loop_normal() 2025-07-20 15:49:19,158 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context 2025-07-20 15:49:19,158 - sglang - INFO - return func(*args, **kwargs) 2025-07-20 15:49:19,158 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 15:49:19,158 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/managers/scheduler.py", line 478, in event_loop_normal 2025-07-20 15:49:19,158 - sglang - INFO - self.process_batch_result(batch, result) 2025-07-20 15:49:19,158 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/managers/scheduler.py", line 1101, in process_batch_result 2025-07-20 15:49:19,158 - sglang - INFO - self.process_batch_result_decode(batch, result) 2025-07-20 15:49:19,158 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/managers/scheduler.py", line 1228, in process_batch_result_decode 2025-07-20 15:49:19,158 - sglang - INFO - next_token_ids = next_token_ids.tolist() 2025-07-20 15:49:19,158 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 15:49:19,158 - sglang - INFO - KeyboardInterrupt 2025-07-20 15:49:19,158 - sglang - INFO - Traceback (most recent call last): 2025-07-20 15:49:19,158 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/multiprocessing/process.py", line 314, in _bootstrap 2025-07-20 15:49:19,158 - sglang - INFO - self.run() 2025-07-20 15:49:19,158 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/multiprocessing/process.py", line 108, in run 2025-07-20 15:49:19,158 - sglang - INFO - self._target(*self._args, **self._kwargs) 2025-07-20 15:49:19,158 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/managers/detokenizer_manager.py", line 240, in run_detokenizer_process 2025-07-20 15:49:19,158 - sglang - INFO - manager.event_loop() 2025-07-20 15:49:19,158 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/sglang/srt/managers/detokenizer_manager.py", line 113, in event_loop 2025-07-20 15:49:19,158 - sglang - INFO - recv_obj = self.recv_from_scheduler.recv_pyobj() 2025-07-20 15:49:19,158 - sglang - INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-20 15:49:19,158 - sglang - INFO - File "/usr/local/miniconda3/envs/olmocr/lib/python3.11/site-packages/zmq/sugar/socket.py", line 989, in recv_pyobj 2025-07-20 15:49:19,158 - sglang - INFO - msg = self.recv(flags) 2025-07-20 15:49:19,158 - sglang - INFO - ^^^^^^^^^^^^^^^^ 2025-07-20 15:49:19,158 - sglang - INFO - File "_zmq.py", line 1147, in zmq.backend.cython._zmq.Socket.recv 2025-07-20 15:49:19,158 - sglang - INFO - File "_zmq.py", line 1182, in zmq.backend.cython._zmq.Socket.recv 2025-07-20 15:49:19,158 - sglang - INFO - File "_zmq.py", line 1337, in zmq.backend.cython._zmq._recv_copy 2025-07-20 15:49:19,158 - sglang - INFO - File "_zmq.py", line 169, in zmq.backend.cython._zmq._check_rc 2025-07-20 15:49:19,158 - sglang - INFO - KeyboardInterrupt 2025-07-20 15:49:19,164 - __main__ - INFO - Got cancellation request for SGLang server 2025-07-20 15:50:09,151 - __main__ - INFO - Got --pdfs argument, going to add to the work queue 2025-07-20 15:50:09,152 - __main__ - INFO - Loading file at tests/gnarly_pdfs/horribleocr.pdf as PDF document 2025-07-20 15:50:09,152 - __main__ - INFO - Found 1 total pdf paths to add 2025-07-20 15:50:09,158 - __main__ - INFO - Calculated items_per_group: 500 based on average pages per PDF: 1.00 2025-07-20 15:50:09,372 - __main__ - INFO - Starting pipeline with PID 599566 2025-07-20 15:50:09,372 - __main__ - INFO - Using local model path at '/root/llm/olmOCR-7B-0225-preview' 2025-07-20 15:50:14,457 - __main__ - WARNING - Attempt 1: Please wait for sglang server to become ready... 2025-07-20 15:50:17,827 - sglang - INFO - [2025-07-20 15:50:17] server_args=ServerArgs(model_path='/root/llm/olmOCR-7B-0225-preview', tokenizer_path='/root/llm/olmOCR-7B-0225-preview', tokenizer_mode='auto', load_format='auto', trust_remote_code=False, dtype='auto', kv_cache_dtype='auto', quantization_param_path=None, quantization=None, context_length=None, device='cuda', served_model_name='/root/llm/olmOCR-7B-0225-preview', chat_template='qwen2-vl', is_embedding=False, revision=None, skip_tokenizer_init=False, host='127.0.0.1', port=30025, mem_fraction_static=0.8, max_running_requests=None, max_total_tokens=None, chunked_prefill_size=2048, max_prefill_tokens=16384, schedule_policy='lpm', schedule_conservativeness=1.0, cpu_offload_gb=0, prefill_only_one_req=False, tp_size=1, stream_interval=1, stream_output=False, random_seed=378345866, constrained_json_whitespace_pattern=None, watchdog_timeout=300, download_dir=None, base_gpu_id=0, log_level='info', log_level_http='warning', log_requests=False, show_time_cost=False, enable_metrics=False, decode_log_interval=40, api_key=None, file_storage_pth='sglang_storage', enable_cache_report=False, dp_size=1, load_balance_method='round_robin', ep_size=1, dist_init_addr=None, nnodes=1, node_rank=0, json_model_override_args='{}', lora_paths=None, max_loras_per_batch=8, attention_backend='flashinfer', sampling_backend='flashinfer', grammar_backend='outlines', speculative_draft_model_path=None, speculative_algorithm=None, speculative_num_steps=5, speculative_num_draft_tokens=64, speculative_eagle_topk=8, enable_double_sparsity=False, ds_channel_config_path=None, ds_heavy_channel_num=32, ds_heavy_token_num=256, ds_heavy_channel_type='qk', ds_sparse_decode_threshold=4096, disable_radix_cache=False, disable_jump_forward=False, disable_cuda_graph=False, disable_cuda_graph_padding=False, disable_outlines_disk_cache=False, disable_custom_all_reduce=False, disable_mla=False, disable_overlap_schedule=False, enable_mixed_chunk=False, enable_dp_attention=False, enable_ep_moe=False, enable_torch_compile=False, torch_compile_max_bs=32, cuda_graph_max_bs=8, cuda_graph_bs=None, torchao_config='', enable_nan_detection=False, enable_p2p_check=False, triton_attention_reduce_in_fp32=False, triton_attention_num_kv_splits=8, num_continuous_decode_steps=1, delete_ckpt_after_loading=False, enable_memory_saver=False, allow_auto_truncate=False, enable_custom_logit_processor=False, tool_call_parser=None) 2025-07-20 15:50:17,828 - __main__ - INFO - [2025-07-20 15:50:17] server_args=ServerArgs(model_path='/root/llm/olmOCR-7B-0225-preview', tokenizer_path='/root/llm/olmOCR-7B-0225-preview', tokenizer_mode='auto', load_format='auto', trust_remote_code=False, dtype='auto', kv_cache_dtype='auto', quantization_param_path=None, quantization=None, context_length=None, device='cuda', served_model_name='/root/llm/olmOCR-7B-0225-preview', chat_template='qwen2-vl', is_embedding=False, revision=None, skip_tokenizer_init=False, host='127.0.0.1', port=30025, mem_fraction_static=0.8, max_running_requests=None, max_total_tokens=None, chunked_prefill_size=2048, max_prefill_tokens=16384, schedule_policy='lpm', schedule_conservativeness=1.0, cpu_offload_gb=0, prefill_only_one_req=False, tp_size=1, stream_interval=1, stream_output=False, random_seed=378345866, constrained_json_whitespace_pattern=None, watchdog_timeout=300, download_dir=None, base_gpu_id=0, log_level='info', log_level_http='warning', log_requests=False, show_time_cost=False, enable_metrics=False, decode_log_interval=40, api_key=None, file_storage_pth='sglang_storage', enable_cache_report=False, dp_size=1, load_balance_method='round_robin', ep_size=1, dist_init_addr=None, nnodes=1, node_rank=0, json_model_override_args='{}', lora_paths=None, max_loras_per_batch=8, attention_backend='flashinfer', sampling_backend='flashinfer', grammar_backend='outlines', speculative_draft_model_path=None, speculative_algorithm=None, speculative_num_steps=5, speculative_num_draft_tokens=64, speculative_eagle_topk=8, enable_double_sparsity=False, ds_channel_config_path=None, ds_heavy_channel_num=32, ds_heavy_token_num=256, ds_heavy_channel_type='qk', ds_sparse_decode_threshold=4096, disable_radix_cache=False, disable_jump_forward=False, disable_cuda_graph=False, disable_cuda_graph_padding=False, disable_outlines_disk_cache=False, disable_custom_all_reduce=False, disable_mla=False, disable_overlap_schedule=False, enable_mixed_chunk=False, enable_dp_attention=False, enable_ep_moe=False, enable_torch_compile=False, torch_compile_max_bs=32, cuda_graph_max_bs=8, cuda_graph_bs=None, torchao_config='', enable_nan_detection=False, enable_p2p_check=False, triton_attention_reduce_in_fp32=False, triton_attention_num_kv_splits=8, num_continuous_decode_steps=1, delete_ckpt_after_loading=False, enable_memory_saver=False, allow_auto_truncate=False, enable_custom_logit_processor=False, tool_call_parser=None) 2025-07-20 15:50:18,989 - sglang - INFO - [2025-07-20 15:50:18] Use chat template for the OpenAI-compatible API server: qwen2-vl 2025-07-20 15:50:18,990 - __main__ - INFO - [2025-07-20 15:50:18] Use chat template for the OpenAI-compatible API server: qwen2-vl 2025-07-20 15:50:20,550 - __main__ - WARNING - Attempt 2: Please wait for sglang server to become ready... 2025-07-20 15:50:24,963 - sglang - INFO - [2025-07-20 15:50:24 TP0] Overlap scheduler is disabled for multimodal models. 2025-07-20 15:50:24,963 - __main__ - INFO - [2025-07-20 15:50:24 TP0] Overlap scheduler is disabled for multimodal models. 2025-07-20 15:50:24,965 - sglang - INFO - [2025-07-20 15:50:24 TP0] Automatically reduce --mem-fraction-static to 0.760 because this is a multimodal model. 2025-07-20 15:50:24,965 - __main__ - INFO - [2025-07-20 15:50:24 TP0] Automatically reduce --mem-fraction-static to 0.760 because this is a multimodal model. 2025-07-20 15:50:24,966 - sglang - INFO - [2025-07-20 15:50:24 TP0] Automatically turn off --chunked-prefill-size and disable radix cache for qwen2-vl. 2025-07-20 15:50:24,966 - __main__ - INFO - [2025-07-20 15:50:24 TP0] Automatically turn off --chunked-prefill-size and disable radix cache for qwen2-vl. 2025-07-20 15:50:24,966 - sglang - INFO - [2025-07-20 15:50:24 TP0] Init torch distributed begin. 2025-07-20 15:50:24,966 - __main__ - INFO - [2025-07-20 15:50:24 TP0] Init torch distributed begin. 2025-07-20 15:50:26,630 - __main__ - WARNING - Attempt 3: Please wait for sglang server to become ready... 2025-07-20 15:50:30,629 - sglang - INFO - [2025-07-20 15:50:30 TP0] Load weight begin. avail mem=23.33 GB 2025-07-20 15:50:30,629 - __main__ - INFO - [2025-07-20 15:50:30 TP0] Load weight begin. avail mem=23.33 GB 2025-07-20 15:50:31,373 - sglang - INFO - Loading safetensors checkpoint shards: 0% Completed | 0/4 [00:00 32768). Running this sequence through the model will result in indexing errors 2025-08-24 23:55:52,933 - __main__ - INFO - Queue remaining: 0 2025-08-24 23:55:52,933 - __main__ - INFO - Metric Name Lifetime (tokens/sec) Recently (tokens/sec) ---------------------------------------------------------------------------------- 2025-08-24 23:55:52,933 - __main__ - INFO - Worker ID | started ----------+-------- 0 | 1 2025-08-24 23:55:58,157 - __main__ - WARNING - ValueError on attempt 0 for ./workspace/map1.pdf-1: - Got BadRequestError from server: b'{"object":"error","message":"The input (78749 tokens) is longer than the model\'s context length (32768 tokens).","type":"BadRequestError","param":null,"code":400}', skipping this response 2025-08-24 23:56:02,935 - __main__ - INFO - Queue remaining: 0 2025-08-24 23:56:02,935 - __main__ - INFO - Metric Name Lifetime (tokens/sec) Recently (tokens/sec) ---------------------------------------------------------------------------------- 2025-08-24 23:56:02,935 - __main__ - INFO - Worker ID | started ----------+-------- 0 | 1 2025-08-24 23:56:03,558 - __main__ - INFO - Semaphore released, allowing a worker to proceed. 2025-08-24 23:56:05,826 - __main__ - INFO - Built page query for ./workspace/map1.pdf-1 2025-08-24 23:56:06,133 - __main__ - WARNING - ValueError on attempt 1 for ./workspace/map1.pdf-1: - Got BadRequestError from server: b'{"object":"error","message":"The input (78749 tokens) is longer than the model\'s context length (32768 tokens).","type":"BadRequestError","param":null,"code":400}', skipping this response 2025-08-24 23:56:10,964 - __main__ - INFO - Built page query for ./workspace/map1.pdf-1 2025-08-24 23:56:11,286 - __main__ - WARNING - ValueError on attempt 2 for ./workspace/map1.pdf-1: - Got BadRequestError from server: b'{"object":"error","message":"The input (78749 tokens) is longer than the model\'s context length (32768 tokens).","type":"BadRequestError","param":null,"code":400}', skipping this response 2025-08-24 23:56:12,936 - __main__ - INFO - Queue remaining: 0 2025-08-24 23:56:12,937 - __main__ - INFO - Metric Name Lifetime (tokens/sec) Recently (tokens/sec) ---------------------------------------------------------------------------------- 2025-08-24 23:56:12,937 - __main__ - INFO - Worker ID | started ----------+-------- 0 | 1 2025-08-24 23:56:15,671 - __main__ - INFO - Built page query for ./workspace/map1.pdf-1 2025-08-24 23:56:16,009 - __main__ - WARNING - ValueError on attempt 3 for ./workspace/map1.pdf-1: - Got BadRequestError from server: b'{"object":"error","message":"The input (78749 tokens) is longer than the model\'s context length (32768 tokens).","type":"BadRequestError","param":null,"code":400}', skipping this response 2025-08-24 23:56:20,410 - __main__ - INFO - Built page query for ./workspace/map1.pdf-1 2025-08-24 23:56:20,768 - __main__ - WARNING - ValueError on attempt 4 for ./workspace/map1.pdf-1: - Got BadRequestError from server: b'{"object":"error","message":"The input (78749 tokens) is longer than the model\'s context length (32768 tokens).","type":"BadRequestError","param":null,"code":400}', skipping this response 2025-08-24 23:56:22,938 - __main__ - INFO - Queue remaining: 0 2025-08-24 23:56:22,938 - __main__ - INFO - Metric Name Lifetime (tokens/sec) Recently (tokens/sec) ---------------------------------------------------------------------------------- 2025-08-24 23:56:22,939 - __main__ - INFO - Worker ID | started ----------+-------- 0 | 1 2025-08-24 23:56:25,382 - __main__ - INFO - Built page query for ./workspace/map1.pdf-1 2025-08-24 23:56:25,713 - __main__ - WARNING - ValueError on attempt 5 for ./workspace/map1.pdf-1: - Got BadRequestError from server: b'{"object":"error","message":"The input (78749 tokens) is longer than the model\'s context length (32768 tokens).","type":"BadRequestError","param":null,"code":400}', skipping this response 2025-08-24 23:56:29,904 - __main__ - INFO - Built page query for ./workspace/map1.pdf-1 2025-08-24 23:56:30,241 - __main__ - WARNING - ValueError on attempt 6 for ./workspace/map1.pdf-1: - Got BadRequestError from server: b'{"object":"error","message":"The input (78749 tokens) is longer than the model\'s context length (32768 tokens).","type":"BadRequestError","param":null,"code":400}', skipping this response 2025-08-24 23:56:32,940 - __main__ - INFO - Queue remaining: 0 2025-08-24 23:56:32,940 - __main__ - INFO - Metric Name Lifetime (tokens/sec) Recently (tokens/sec) ---------------------------------------------------------------------------------- 2025-08-24 23:56:32,940 - __main__ - INFO - Worker ID | started ----------+-------- 0 | 1 2025-08-24 23:56:34,598 - __main__ - INFO - Built page query for ./workspace/map1.pdf-1 2025-08-24 23:56:34,937 - __main__ - WARNING - ValueError on attempt 7 for ./workspace/map1.pdf-1: - Got BadRequestError from server: b'{"object":"error","message":"The input (78749 tokens) is longer than the model\'s context length (32768 tokens).","type":"BadRequestError","param":null,"code":400}', skipping this response 2025-08-24 23:56:34,937 - __main__ - ERROR - Failed to process ./workspace/map1.pdf-1 after 8 attempts. 2025-08-24 23:56:35,309 - __main__ - ERROR - Document ./workspace/map1.pdf has 1 fallback pages out of 1 exceeding max_page_error_rate of 0.004, discarding document. 2025-08-24 23:56:35,310 - __main__ - INFO - Finished TaskGroup for worker on 064eedd4edcd817030605d106353694b3e3ec8b1 2025-08-24 23:56:35,310 - __main__ - INFO - Got 0 docs for 064eedd4edcd817030605d106353694b3e3ec8b1 2025-08-24 23:56:35,311 - __main__ - INFO - Worker 0 exiting due to empty queue 2025-08-24 23:56:35,312 - __main__ - INFO - Work done 2025-08-24 23:56:35,312 - __main__ - INFO - Got cancellation request for SGLang server 2025-08-24 23:57:04,821 - __main__ - INFO - Got --pdfs argument, going to add to the work queue 2025-08-24 23:57:04,821 - __main__ - INFO - Loading file at ./workspace/map1.pdf as PDF document 2025-08-24 23:57:04,821 - __main__ - INFO - Found 1 total pdf paths to add 2025-08-24 23:57:04,825 - __main__ - INFO - Calculated items_per_group: 10 based on average pages per PDF: 1.00 2025-08-24 23:57:05,000 - __main__ - INFO - Starting pipeline with PID 482844 2025-08-24 23:57:05,000 - __main__ - INFO - Using local model path at '/root/llm/olmOCR-7B-0225-preview' 2025-08-24 23:57:05,073 - __main__ - WARNING - Attempt 1: Please wait for sglang server to become ready... 2025-08-24 23:57:06,104 - __main__ - WARNING - Attempt 2: Please wait for sglang server to become ready... 2025-08-24 23:57:07,161 - __main__ - WARNING - Attempt 3: Please wait for sglang server to become ready... 2025-08-24 23:57:08,218 - __main__ - WARNING - Attempt 4: Please wait for sglang server to become ready... 2025-08-24 23:57:09,267 - __main__ - WARNING - Attempt 5: Please wait for sglang server to become ready... 2025-08-24 23:57:10,371 - __main__ - WARNING - Attempt 6: Please wait for sglang server to become ready... 2025-08-24 23:57:11,124 - sglang - INFO - [2025-08-24 23:57:11] server_args=ServerArgs(model_path='/root/llm/olmOCR-7B-0225-preview', tokenizer_path='/root/llm/olmOCR-7B-0225-preview', tokenizer_mode='auto', load_format='auto', trust_remote_code=False, dtype='auto', kv_cache_dtype='auto', quantization_param_path=None, quantization=None, context_length=None, device='cuda', served_model_name='/root/llm/olmOCR-7B-0225-preview', chat_template='qwen2-vl', is_embedding=False, revision=None, skip_tokenizer_init=False, host='127.0.0.1', port=30026, mem_fraction_static=0.8, max_running_requests=None, max_total_tokens=None, chunked_prefill_size=2048, max_prefill_tokens=16384, schedule_policy='lpm', schedule_conservativeness=1.0, cpu_offload_gb=0, prefill_only_one_req=False, tp_size=1, stream_interval=1, stream_output=False, random_seed=608298291, constrained_json_whitespace_pattern=None, watchdog_timeout=300, download_dir=None, base_gpu_id=0, log_level='info', log_level_http='warning', log_requests=False, show_time_cost=False, enable_metrics=False, decode_log_interval=40, api_key=None, file_storage_pth='sglang_storage', enable_cache_report=False, dp_size=1, load_balance_method='round_robin', ep_size=1, dist_init_addr=None, nnodes=1, node_rank=0, json_model_override_args='{}', lora_paths=None, max_loras_per_batch=8, attention_backend='flashinfer', sampling_backend='flashinfer', grammar_backend='outlines', speculative_draft_model_path=None, speculative_algorithm=None, speculative_num_steps=5, speculative_num_draft_tokens=64, speculative_eagle_topk=8, enable_double_sparsity=False, ds_channel_config_path=None, ds_heavy_channel_num=32, ds_heavy_token_num=256, ds_heavy_channel_type='qk', ds_sparse_decode_threshold=4096, disable_radix_cache=False, disable_jump_forward=False, disable_cuda_graph=False, disable_cuda_graph_padding=False, disable_outlines_disk_cache=False, disable_custom_all_reduce=False, disable_mla=False, disable_overlap_schedule=False, enable_mixed_chunk=False, enable_dp_attention=False, enable_ep_moe=False, enable_torch_compile=False, torch_compile_max_bs=32, cuda_graph_max_bs=8, cuda_graph_bs=None, torchao_config='', enable_nan_detection=False, enable_p2p_check=False, triton_attention_reduce_in_fp32=False, triton_attention_num_kv_splits=8, num_continuous_decode_steps=1, delete_ckpt_after_loading=False, enable_memory_saver=False, allow_auto_truncate=False, enable_custom_logit_processor=False, tool_call_parser=None) 2025-08-24 23:57:11,124 - __main__ - INFO - [2025-08-24 23:57:11] server_args=ServerArgs(model_path='/root/llm/olmOCR-7B-0225-preview', tokenizer_path='/root/llm/olmOCR-7B-0225-preview', tokenizer_mode='auto', load_format='auto', trust_remote_code=False, dtype='auto', kv_cache_dtype='auto', quantization_param_path=None, quantization=None, context_length=None, device='cuda', served_model_name='/root/llm/olmOCR-7B-0225-preview', chat_template='qwen2-vl', is_embedding=False, revision=None, skip_tokenizer_init=False, host='127.0.0.1', port=30026, mem_fraction_static=0.8, max_running_requests=None, max_total_tokens=None, chunked_prefill_size=2048, max_prefill_tokens=16384, schedule_policy='lpm', schedule_conservativeness=1.0, cpu_offload_gb=0, prefill_only_one_req=False, tp_size=1, stream_interval=1, stream_output=False, random_seed=608298291, constrained_json_whitespace_pattern=None, watchdog_timeout=300, download_dir=None, base_gpu_id=0, log_level='info', log_level_http='warning', log_requests=False, show_time_cost=False, enable_metrics=False, decode_log_interval=40, api_key=None, file_storage_pth='sglang_storage', enable_cache_report=False, dp_size=1, load_balance_method='round_robin', ep_size=1, dist_init_addr=None, nnodes=1, node_rank=0, json_model_override_args='{}', lora_paths=None, max_loras_per_batch=8, attention_backend='flashinfer', sampling_backend='flashinfer', grammar_backend='outlines', speculative_draft_model_path=None, speculative_algorithm=None, speculative_num_steps=5, speculative_num_draft_tokens=64, speculative_eagle_topk=8, enable_double_sparsity=False, ds_channel_config_path=None, ds_heavy_channel_num=32, ds_heavy_token_num=256, ds_heavy_channel_type='qk', ds_sparse_decode_threshold=4096, disable_radix_cache=False, disable_jump_forward=False, disable_cuda_graph=False, disable_cuda_graph_padding=False, disable_outlines_disk_cache=False, disable_custom_all_reduce=False, disable_mla=False, disable_overlap_schedule=False, enable_mixed_chunk=False, enable_dp_attention=False, enable_ep_moe=False, enable_torch_compile=False, torch_compile_max_bs=32, cuda_graph_max_bs=8, cuda_graph_bs=None, torchao_config='', enable_nan_detection=False, enable_p2p_check=False, triton_attention_reduce_in_fp32=False, triton_attention_num_kv_splits=8, num_continuous_decode_steps=1, delete_ckpt_after_loading=False, enable_memory_saver=False, allow_auto_truncate=False, enable_custom_logit_processor=False, tool_call_parser=None) 2025-08-24 23:57:11,431 - __main__ - WARNING - Attempt 7: Please wait for sglang server to become ready... 2025-08-24 23:57:12,069 - sglang - INFO - [2025-08-24 23:57:12] Use chat template for the OpenAI-compatible API server: qwen2-vl 2025-08-24 23:57:12,069 - __main__ - INFO - [2025-08-24 23:57:12] Use chat template for the OpenAI-compatible API server: qwen2-vl 2025-08-24 23:57:12,504 - __main__ - WARNING - Attempt 8: Please wait for sglang server to become ready... 2025-08-24 23:57:13,550 - __main__ - WARNING - Attempt 9: Please wait for sglang server to become ready... 2025-08-24 23:57:14,596 - __main__ - WARNING - Attempt 10: Please wait for sglang server to become ready... 2025-08-24 23:57:15,643 - __main__ - WARNING - Attempt 11: Please wait for sglang server to become ready... 2025-08-24 23:57:16,689 - __main__ - WARNING - Attempt 12: Please wait for sglang server to become ready... 2025-08-24 23:57:17,735 - __main__ - WARNING - Attempt 13: Please wait for sglang server to become ready... 2025-08-24 23:57:18,780 - __main__ - WARNING - Attempt 14: Please wait for sglang server to become ready... 2025-08-24 23:57:19,094 - sglang - INFO - [2025-08-24 23:57:19 TP0] Overlap scheduler is disabled for multimodal models. 2025-08-24 23:57:19,094 - __main__ - INFO - [2025-08-24 23:57:19 TP0] Overlap scheduler is disabled for multimodal models. 2025-08-24 23:57:19,097 - sglang - INFO - [2025-08-24 23:57:19 TP0] Automatically reduce --mem-fraction-static to 0.760 because this is a multimodal model. 2025-08-24 23:57:19,097 - __main__ - INFO - [2025-08-24 23:57:19 TP0] Automatically reduce --mem-fraction-static to 0.760 because this is a multimodal model. 2025-08-24 23:57:19,097 - sglang - INFO - [2025-08-24 23:57:19 TP0] Automatically turn off --chunked-prefill-size and disable radix cache for qwen2-vl. 2025-08-24 23:57:19,097 - __main__ - INFO - [2025-08-24 23:57:19 TP0] Automatically turn off --chunked-prefill-size and disable radix cache for qwen2-vl. 2025-08-24 23:57:19,098 - sglang - INFO - [2025-08-24 23:57:19 TP0] Init torch distributed begin. 2025-08-24 23:57:19,098 - __main__ - INFO - [2025-08-24 23:57:19 TP0] Init torch distributed begin. 2025-08-24 23:57:19,859 - __main__ - WARNING - Attempt 15: Please wait for sglang server to become ready... 2025-08-24 23:57:20,488 - __main__ - INFO - Got cancellation request for SGLang server 2025-08-24 23:57:42,899 - __main__ - INFO - Got --pdfs argument, going to add to the work queue 2025-08-24 23:57:42,900 - __main__ - INFO - Loading file at ./workspace/map1.pdf as PDF document 2025-08-24 23:57:42,900 - __main__ - INFO - Found 1 total pdf paths to add 2025-08-24 23:57:42,904 - __main__ - INFO - Calculated items_per_group: 10 based on average pages per PDF: 1.00 2025-08-24 23:57:43,097 - __main__ - INFO - Starting pipeline with PID 483718 2025-08-24 23:57:43,097 - __main__ - INFO - Using local model path at '/root/llm/olmOCR-7B-0225-preview' 2025-08-24 23:57:43,196 - __main__ - WARNING - Attempt 1: Please wait for sglang server to become ready... 2025-08-24 23:57:44,232 - __main__ - WARNING - Attempt 2: Please wait for sglang server to become ready... 2025-08-24 23:57:45,288 - __main__ - WARNING - Attempt 3: Please wait for sglang server to become ready... 2025-08-24 23:57:46,333 - __main__ - WARNING - Attempt 4: Please wait for sglang server to become ready... 2025-08-24 23:57:47,364 - __main__ - WARNING - Attempt 5: Please wait for sglang server to become ready... 2025-08-24 23:57:48,478 - __main__ - WARNING - Attempt 6: Please wait for sglang server to become ready... 2025-08-24 23:57:49,539 - __main__ - WARNING - Attempt 7: Please wait for sglang server to become ready... 2025-08-24 23:57:49,692 - sglang - INFO - [2025-08-24 23:57:49] server_args=ServerArgs(model_path='/root/llm/olmOCR-7B-0225-preview', tokenizer_path='/root/llm/olmOCR-7B-0225-preview', tokenizer_mode='auto', load_format='auto', trust_remote_code=False, dtype='auto', kv_cache_dtype='auto', quantization_param_path=None, quantization=None, context_length=None, device='cuda', served_model_name='/root/llm/olmOCR-7B-0225-preview', chat_template='qwen2-vl', is_embedding=False, revision=None, skip_tokenizer_init=False, host='127.0.0.1', port=30026, mem_fraction_static=0.8, max_running_requests=None, max_total_tokens=None, chunked_prefill_size=2048, max_prefill_tokens=16384, schedule_policy='lpm', schedule_conservativeness=1.0, cpu_offload_gb=0, prefill_only_one_req=False, tp_size=1, stream_interval=1, stream_output=False, random_seed=1010487791, constrained_json_whitespace_pattern=None, watchdog_timeout=300, download_dir=None, base_gpu_id=0, log_level='info', log_level_http='warning', log_requests=False, show_time_cost=False, enable_metrics=False, decode_log_interval=40, api_key=None, file_storage_pth='sglang_storage', enable_cache_report=False, dp_size=1, load_balance_method='round_robin', ep_size=1, dist_init_addr=None, nnodes=1, node_rank=0, json_model_override_args='{}', lora_paths=None, max_loras_per_batch=8, attention_backend='flashinfer', sampling_backend='flashinfer', grammar_backend='outlines', speculative_draft_model_path=None, speculative_algorithm=None, speculative_num_steps=5, speculative_num_draft_tokens=64, speculative_eagle_topk=8, enable_double_sparsity=False, ds_channel_config_path=None, ds_heavy_channel_num=32, ds_heavy_token_num=256, ds_heavy_channel_type='qk', ds_sparse_decode_threshold=4096, disable_radix_cache=False, disable_jump_forward=False, disable_cuda_graph=False, disable_cuda_graph_padding=False, disable_outlines_disk_cache=False, disable_custom_all_reduce=False, disable_mla=False, disable_overlap_schedule=False, enable_mixed_chunk=False, enable_dp_attention=False, enable_ep_moe=False, enable_torch_compile=False, torch_compile_max_bs=32, cuda_graph_max_bs=8, cuda_graph_bs=None, torchao_config='', enable_nan_detection=False, enable_p2p_check=False, triton_attention_reduce_in_fp32=False, triton_attention_num_kv_splits=8, num_continuous_decode_steps=1, delete_ckpt_after_loading=False, enable_memory_saver=False, allow_auto_truncate=False, enable_custom_logit_processor=False, tool_call_parser=None) 2025-08-24 23:57:49,692 - __main__ - INFO - [2025-08-24 23:57:49] server_args=ServerArgs(model_path='/root/llm/olmOCR-7B-0225-preview', tokenizer_path='/root/llm/olmOCR-7B-0225-preview', tokenizer_mode='auto', load_format='auto', trust_remote_code=False, dtype='auto', kv_cache_dtype='auto', quantization_param_path=None, quantization=None, context_length=None, device='cuda', served_model_name='/root/llm/olmOCR-7B-0225-preview', chat_template='qwen2-vl', is_embedding=False, revision=None, skip_tokenizer_init=False, host='127.0.0.1', port=30026, mem_fraction_static=0.8, max_running_requests=None, max_total_tokens=None, chunked_prefill_size=2048, max_prefill_tokens=16384, schedule_policy='lpm', schedule_conservativeness=1.0, cpu_offload_gb=0, prefill_only_one_req=False, tp_size=1, stream_interval=1, stream_output=False, random_seed=1010487791, constrained_json_whitespace_pattern=None, watchdog_timeout=300, download_dir=None, base_gpu_id=0, log_level='info', log_level_http='warning', log_requests=False, show_time_cost=False, enable_metrics=False, decode_log_interval=40, api_key=None, file_storage_pth='sglang_storage', enable_cache_report=False, dp_size=1, load_balance_method='round_robin', ep_size=1, dist_init_addr=None, nnodes=1, node_rank=0, json_model_override_args='{}', lora_paths=None, max_loras_per_batch=8, attention_backend='flashinfer', sampling_backend='flashinfer', grammar_backend='outlines', speculative_draft_model_path=None, speculative_algorithm=None, speculative_num_steps=5, speculative_num_draft_tokens=64, speculative_eagle_topk=8, enable_double_sparsity=False, ds_channel_config_path=None, ds_heavy_channel_num=32, ds_heavy_token_num=256, ds_heavy_channel_type='qk', ds_sparse_decode_threshold=4096, disable_radix_cache=False, disable_jump_forward=False, disable_cuda_graph=False, disable_cuda_graph_padding=False, disable_outlines_disk_cache=False, disable_custom_all_reduce=False, disable_mla=False, disable_overlap_schedule=False, enable_mixed_chunk=False, enable_dp_attention=False, enable_ep_moe=False, enable_torch_compile=False, torch_compile_max_bs=32, cuda_graph_max_bs=8, cuda_graph_bs=None, torchao_config='', enable_nan_detection=False, enable_p2p_check=False, triton_attention_reduce_in_fp32=False, triton_attention_num_kv_splits=8, num_continuous_decode_steps=1, delete_ckpt_after_loading=False, enable_memory_saver=False, allow_auto_truncate=False, enable_custom_logit_processor=False, tool_call_parser=None) 2025-08-24 23:57:50,619 - __main__ - WARNING - Attempt 8: Please wait for sglang server to become ready... 2025-08-24 23:57:50,815 - sglang - INFO - [2025-08-24 23:57:50] Use chat template for the OpenAI-compatible API server: qwen2-vl 2025-08-24 23:57:50,816 - __main__ - INFO - [2025-08-24 23:57:50] Use chat template for the OpenAI-compatible API server: qwen2-vl 2025-08-24 23:57:51,700 - __main__ - WARNING - Attempt 9: Please wait for sglang server to become ready... 2025-08-24 23:57:52,767 - __main__ - WARNING - Attempt 10: Please wait for sglang server to become ready... 2025-08-24 23:57:53,831 - __main__ - WARNING - Attempt 11: Please wait for sglang server to become ready... 2025-08-24 23:57:54,956 - __main__ - WARNING - Attempt 12: Please wait for sglang server to become ready... 2025-08-24 23:57:56,012 - __main__ - WARNING - Attempt 13: Please wait for sglang server to become ready... 2025-08-24 23:57:57,079 - __main__ - WARNING - Attempt 14: Please wait for sglang server to become ready... 2025-08-24 23:57:57,540 - sglang - INFO - [2025-08-24 23:57:57 TP0] Overlap scheduler is disabled for multimodal models. 2025-08-24 23:57:57,540 - __main__ - INFO - [2025-08-24 23:57:57 TP0] Overlap scheduler is disabled for multimodal models. 2025-08-24 23:57:57,544 - sglang - INFO - [2025-08-24 23:57:57 TP0] Automatically reduce --mem-fraction-static to 0.760 because this is a multimodal model. 2025-08-24 23:57:57,544 - __main__ - INFO - [2025-08-24 23:57:57 TP0] Automatically reduce --mem-fraction-static to 0.760 because this is a multimodal model. 2025-08-24 23:57:57,544 - sglang - INFO - [2025-08-24 23:57:57 TP0] Automatically turn off --chunked-prefill-size and disable radix cache for qwen2-vl. 2025-08-24 23:57:57,544 - __main__ - INFO - [2025-08-24 23:57:57 TP0] Automatically turn off --chunked-prefill-size and disable radix cache for qwen2-vl. 2025-08-24 23:57:57,544 - sglang - INFO - [2025-08-24 23:57:57 TP0] Init torch distributed begin. 2025-08-24 23:57:57,544 - __main__ - INFO - [2025-08-24 23:57:57 TP0] Init torch distributed begin. 2025-08-24 23:57:58,147 - __main__ - WARNING - Attempt 15: Please wait for sglang server to become ready... 2025-08-24 23:57:59,210 - __main__ - WARNING - Attempt 16: Please wait for sglang server to become ready... 2025-08-24 23:58:00,268 - __main__ - WARNING - Attempt 17: Please wait for sglang server to become ready... 2025-08-24 23:58:01,321 - __main__ - WARNING - Attempt 18: Please wait for sglang server to become ready... 2025-08-24 23:58:02,367 - __main__ - WARNING - Attempt 19: Please wait for sglang server to become ready... 2025-08-24 23:58:02,851 - sglang - INFO - [2025-08-24 23:58:02 TP0] Load weight begin. avail mem=23.33 GB 2025-08-24 23:58:02,851 - __main__ - INFO - [2025-08-24 23:58:02 TP0] Load weight begin. avail mem=23.33 GB 2025-08-24 23:58:03,420 - sglang - INFO - Loading safetensors checkpoint shards: 0% Completed | 0/4 [00:00 32768). Running this sequence through the model will result in indexing errors 2025-08-24 23:58:29,733 - __main__ - INFO - Queue remaining: 0 2025-08-24 23:58:29,733 - __main__ - INFO - Metric Name Lifetime (tokens/sec) Recently (tokens/sec) ---------------------------------------------------------------------------------- 2025-08-24 23:58:29,733 - __main__ - INFO - Worker ID | started ----------+-------- 0 | 1 2025-08-24 23:58:34,535 - __main__ - WARNING - ValueError on attempt 0 for ./workspace/map1.pdf-1: - Got BadRequestError from server: b'{"object":"error","message":"The input (78749 tokens) is longer than the model\'s context length (32768 tokens).","type":"BadRequestError","param":null,"code":400}', skipping this response 2025-08-24 23:58:39,734 - __main__ - INFO - Queue remaining: 0 2025-08-24 23:58:39,735 - __main__ - INFO - Metric Name Lifetime (tokens/sec) Recently (tokens/sec) ---------------------------------------------------------------------------------- 2025-08-24 23:58:39,735 - __main__ - INFO - Worker ID | started ----------+-------- 0 | 1 2025-08-24 23:58:41,440 - __main__ - INFO - Semaphore released, allowing a worker to proceed. 2025-08-24 23:58:42,288 - __main__ - INFO - Built page query for ./workspace/map1.pdf-1 2025-08-24 23:58:42,596 - __main__ - WARNING - ValueError on attempt 1 for ./workspace/map1.pdf-1: - Got BadRequestError from server: b'{"object":"error","message":"The input (78749 tokens) is longer than the model\'s context length (32768 tokens).","type":"BadRequestError","param":null,"code":400}', skipping this response 2025-08-24 23:58:47,737 - __main__ - INFO - Built page query for ./workspace/map1.pdf-1 2025-08-24 23:58:48,060 - __main__ - WARNING - ValueError on attempt 2 for ./workspace/map1.pdf-1: - Got BadRequestError from server: b'{"object":"error","message":"The input (78749 tokens) is longer than the model\'s context length (32768 tokens).","type":"BadRequestError","param":null,"code":400}', skipping this response 2025-08-24 23:58:49,737 - __main__ - INFO - Queue remaining: 0 2025-08-24 23:58:49,738 - __main__ - INFO - Metric Name Lifetime (tokens/sec) Recently (tokens/sec) ---------------------------------------------------------------------------------- 2025-08-24 23:58:49,738 - __main__ - INFO - Worker ID | started ----------+-------- 0 | 1 2025-08-24 23:58:52,705 - __main__ - INFO - Built page query for ./workspace/map1.pdf-1 2025-08-24 23:58:53,022 - __main__ - WARNING - ValueError on attempt 3 for ./workspace/map1.pdf-1: - Got BadRequestError from server: b'{"object":"error","message":"The input (78749 tokens) is longer than the model\'s context length (32768 tokens).","type":"BadRequestError","param":null,"code":400}', skipping this response 2025-08-24 23:58:57,211 - __main__ - INFO - Built page query for ./workspace/map1.pdf-1 2025-08-24 23:58:57,582 - __main__ - WARNING - ValueError on attempt 4 for ./workspace/map1.pdf-1: - Got BadRequestError from server: b'{"object":"error","message":"The input (78749 tokens) is longer than the model\'s context length (32768 tokens).","type":"BadRequestError","param":null,"code":400}', skipping this response 2025-08-24 23:58:59,739 - __main__ - INFO - Queue remaining: 0 2025-08-24 23:58:59,739 - __main__ - INFO - Metric Name Lifetime (tokens/sec) Recently (tokens/sec) ---------------------------------------------------------------------------------- 2025-08-24 23:58:59,739 - __main__ - INFO - Worker ID | started ----------+-------- 0 | 1 2025-08-24 23:59:01,984 - __main__ - INFO - Built page query for ./workspace/map1.pdf-1 2025-08-24 23:59:02,323 - __main__ - WARNING - ValueError on attempt 5 for ./workspace/map1.pdf-1: - Got BadRequestError from server: b'{"object":"error","message":"The input (78749 tokens) is longer than the model\'s context length (32768 tokens).","type":"BadRequestError","param":null,"code":400}', skipping this response 2025-08-24 23:59:06,369 - __main__ - INFO - Built page query for ./workspace/map1.pdf-1 2025-08-24 23:59:06,681 - __main__ - WARNING - ValueError on attempt 6 for ./workspace/map1.pdf-1: - Got BadRequestError from server: b'{"object":"error","message":"The input (78749 tokens) is longer than the model\'s context length (32768 tokens).","type":"BadRequestError","param":null,"code":400}', skipping this response 2025-08-24 23:59:09,741 - __main__ - INFO - Queue remaining: 0 2025-08-24 23:59:09,741 - __main__ - INFO - Metric Name Lifetime (tokens/sec) Recently (tokens/sec) ---------------------------------------------------------------------------------- 2025-08-24 23:59:09,741 - __main__ - INFO - Worker ID | started ----------+-------- 0 | 1 2025-08-24 23:59:10,935 - __main__ - INFO - Built page query for ./workspace/map1.pdf-1 2025-08-24 23:59:11,295 - __main__ - WARNING - ValueError on attempt 7 for ./workspace/map1.pdf-1: - Got BadRequestError from server: b'{"object":"error","message":"The input (78749 tokens) is longer than the model\'s context length (32768 tokens).","type":"BadRequestError","param":null,"code":400}', skipping this response 2025-08-24 23:59:11,296 - __main__ - ERROR - Failed to process ./workspace/map1.pdf-1 after 8 attempts. 2025-08-24 23:59:11,666 - __main__ - ERROR - Document ./workspace/map1.pdf has 1 fallback pages out of 1 exceeding max_page_error_rate of 0.004, discarding document. 2025-08-24 23:59:11,667 - __main__ - INFO - Finished TaskGroup for worker on 064eedd4edcd817030605d106353694b3e3ec8b1 2025-08-24 23:59:11,667 - __main__ - INFO - Got 0 docs for 064eedd4edcd817030605d106353694b3e3ec8b1 2025-08-24 23:59:11,668 - __main__ - INFO - Worker 0 exiting due to empty queue 2025-08-24 23:59:11,668 - __main__ - INFO - Work done 2025-08-24 23:59:11,669 - __main__ - INFO - Got cancellation request for SGLang server 2025-08-24 23:59:37,224 - __main__ - INFO - Got --pdfs argument, going to add to the work queue 2025-08-24 23:59:37,224 - __main__ - INFO - Loading file at ./workspace/UNETR.pdf as PDF document 2025-08-24 23:59:37,224 - __main__ - INFO - Found 1 total pdf paths to add 2025-08-24 23:59:37,230 - __main__ - INFO - Calculated items_per_group: 1 based on average pages per PDF: 11.00 2025-08-24 23:59:37,413 - __main__ - INFO - Starting pipeline with PID 484898 2025-08-24 23:59:37,413 - __main__ - INFO - Using local model path at '/root/llm/olmOCR-7B-0225-preview' 2025-08-24 23:59:37,499 - __main__ - WARNING - Attempt 1: Please wait for sglang server to become ready... 2025-08-24 23:59:38,535 - __main__ - WARNING - Attempt 2: Please wait for sglang server to become ready... 2025-08-24 23:59:39,594 - __main__ - WARNING - Attempt 3: Please wait for sglang server to become ready... 2025-08-24 23:59:40,666 - __main__ - WARNING - Attempt 4: Please wait for sglang server to become ready... 2025-08-24 23:59:41,733 - __main__ - WARNING - Attempt 5: Please wait for sglang server to become ready... 2025-08-24 23:59:42,799 - __main__ - WARNING - Attempt 6: Please wait for sglang server to become ready... 2025-08-24 23:59:43,852 - __main__ - WARNING - Attempt 7: Please wait for sglang server to become ready... 2025-08-24 23:59:44,133 - sglang - INFO - [2025-08-24 23:59:44] server_args=ServerArgs(model_path='/root/llm/olmOCR-7B-0225-preview', tokenizer_path='/root/llm/olmOCR-7B-0225-preview', tokenizer_mode='auto', load_format='auto', trust_remote_code=False, dtype='auto', kv_cache_dtype='auto', quantization_param_path=None, quantization=None, context_length=None, device='cuda', served_model_name='/root/llm/olmOCR-7B-0225-preview', chat_template='qwen2-vl', is_embedding=False, revision=None, skip_tokenizer_init=False, host='127.0.0.1', port=30026, mem_fraction_static=0.8, max_running_requests=None, max_total_tokens=None, chunked_prefill_size=2048, max_prefill_tokens=16384, schedule_policy='lpm', schedule_conservativeness=1.0, cpu_offload_gb=0, prefill_only_one_req=False, tp_size=1, stream_interval=1, stream_output=False, random_seed=1008456358, constrained_json_whitespace_pattern=None, watchdog_timeout=300, download_dir=None, base_gpu_id=0, log_level='info', log_level_http='warning', log_requests=False, show_time_cost=False, enable_metrics=False, decode_log_interval=40, api_key=None, file_storage_pth='sglang_storage', enable_cache_report=False, dp_size=1, load_balance_method='round_robin', ep_size=1, dist_init_addr=None, nnodes=1, node_rank=0, json_model_override_args='{}', lora_paths=None, max_loras_per_batch=8, attention_backend='flashinfer', sampling_backend='flashinfer', grammar_backend='outlines', speculative_draft_model_path=None, speculative_algorithm=None, speculative_num_steps=5, speculative_num_draft_tokens=64, speculative_eagle_topk=8, enable_double_sparsity=False, ds_channel_config_path=None, ds_heavy_channel_num=32, ds_heavy_token_num=256, ds_heavy_channel_type='qk', ds_sparse_decode_threshold=4096, disable_radix_cache=False, disable_jump_forward=False, disable_cuda_graph=False, disable_cuda_graph_padding=False, disable_outlines_disk_cache=False, disable_custom_all_reduce=False, disable_mla=False, disable_overlap_schedule=False, enable_mixed_chunk=False, enable_dp_attention=False, enable_ep_moe=False, enable_torch_compile=False, torch_compile_max_bs=32, cuda_graph_max_bs=8, cuda_graph_bs=None, torchao_config='', enable_nan_detection=False, enable_p2p_check=False, triton_attention_reduce_in_fp32=False, triton_attention_num_kv_splits=8, num_continuous_decode_steps=1, delete_ckpt_after_loading=False, enable_memory_saver=False, allow_auto_truncate=False, enable_custom_logit_processor=False, tool_call_parser=None) 2025-08-24 23:59:44,133 - __main__ - INFO - [2025-08-24 23:59:44] server_args=ServerArgs(model_path='/root/llm/olmOCR-7B-0225-preview', tokenizer_path='/root/llm/olmOCR-7B-0225-preview', tokenizer_mode='auto', load_format='auto', trust_remote_code=False, dtype='auto', kv_cache_dtype='auto', quantization_param_path=None, quantization=None, context_length=None, device='cuda', served_model_name='/root/llm/olmOCR-7B-0225-preview', chat_template='qwen2-vl', is_embedding=False, revision=None, skip_tokenizer_init=False, host='127.0.0.1', port=30026, mem_fraction_static=0.8, max_running_requests=None, max_total_tokens=None, chunked_prefill_size=2048, max_prefill_tokens=16384, schedule_policy='lpm', schedule_conservativeness=1.0, cpu_offload_gb=0, prefill_only_one_req=False, tp_size=1, stream_interval=1, stream_output=False, random_seed=1008456358, constrained_json_whitespace_pattern=None, watchdog_timeout=300, download_dir=None, base_gpu_id=0, log_level='info', log_level_http='warning', log_requests=False, show_time_cost=False, enable_metrics=False, decode_log_interval=40, api_key=None, file_storage_pth='sglang_storage', enable_cache_report=False, dp_size=1, load_balance_method='round_robin', ep_size=1, dist_init_addr=None, nnodes=1, node_rank=0, json_model_override_args='{}', lora_paths=None, max_loras_per_batch=8, attention_backend='flashinfer', sampling_backend='flashinfer', grammar_backend='outlines', speculative_draft_model_path=None, speculative_algorithm=None, speculative_num_steps=5, speculative_num_draft_tokens=64, speculative_eagle_topk=8, enable_double_sparsity=False, ds_channel_config_path=None, ds_heavy_channel_num=32, ds_heavy_token_num=256, ds_heavy_channel_type='qk', ds_sparse_decode_threshold=4096, disable_radix_cache=False, disable_jump_forward=False, disable_cuda_graph=False, disable_cuda_graph_padding=False, disable_outlines_disk_cache=False, disable_custom_all_reduce=False, disable_mla=False, disable_overlap_schedule=False, enable_mixed_chunk=False, enable_dp_attention=False, enable_ep_moe=False, enable_torch_compile=False, torch_compile_max_bs=32, cuda_graph_max_bs=8, cuda_graph_bs=None, torchao_config='', enable_nan_detection=False, enable_p2p_check=False, triton_attention_reduce_in_fp32=False, triton_attention_num_kv_splits=8, num_continuous_decode_steps=1, delete_ckpt_after_loading=False, enable_memory_saver=False, allow_auto_truncate=False, enable_custom_logit_processor=False, tool_call_parser=None) 2025-08-24 23:59:44,921 - __main__ - WARNING - Attempt 8: Please wait for sglang server to become ready... 2025-08-24 23:59:45,150 - sglang - INFO - [2025-08-24 23:59:45] Use chat template for the OpenAI-compatible API server: qwen2-vl 2025-08-24 23:59:45,151 - __main__ - INFO - [2025-08-24 23:59:45] Use chat template for the OpenAI-compatible API server: qwen2-vl 2025-08-24 23:59:45,968 - __main__ - WARNING - Attempt 9: Please wait for sglang server to become ready... 2025-08-24 23:59:47,035 - __main__ - WARNING - Attempt 10: Please wait for sglang server to become ready... 2025-08-24 23:59:48,100 - __main__ - WARNING - Attempt 11: Please wait for sglang server to become ready... 2025-08-24 23:59:49,173 - __main__ - WARNING - Attempt 12: Please wait for sglang server to become ready... 2025-08-24 23:59:50,243 - __main__ - WARNING - Attempt 13: Please wait for sglang server to become ready... 2025-08-24 23:59:51,314 - __main__ - WARNING - Attempt 14: Please wait for sglang server to become ready... 2025-08-24 23:59:51,579 - sglang - INFO - [2025-08-24 23:59:51 TP0] Overlap scheduler is disabled for multimodal models. 2025-08-24 23:59:51,579 - __main__ - INFO - [2025-08-24 23:59:51 TP0] Overlap scheduler is disabled for multimodal models. 2025-08-24 23:59:51,581 - sglang - INFO - [2025-08-24 23:59:51 TP0] Automatically reduce --mem-fraction-static to 0.760 because this is a multimodal model. 2025-08-24 23:59:51,581 - __main__ - INFO - [2025-08-24 23:59:51 TP0] Automatically reduce --mem-fraction-static to 0.760 because this is a multimodal model. 2025-08-24 23:59:51,581 - sglang - INFO - [2025-08-24 23:59:51 TP0] Automatically turn off --chunked-prefill-size and disable radix cache for qwen2-vl. 2025-08-24 23:59:51,581 - __main__ - INFO - [2025-08-24 23:59:51 TP0] Automatically turn off --chunked-prefill-size and disable radix cache for qwen2-vl. 2025-08-24 23:59:51,581 - sglang - INFO - [2025-08-24 23:59:51 TP0] Init torch distributed begin. 2025-08-24 23:59:51,581 - __main__ - INFO - [2025-08-24 23:59:51 TP0] Init torch distributed begin. 2025-08-24 23:59:52,346 - __main__ - WARNING - Attempt 15: Please wait for sglang server to become ready... 2025-08-24 23:59:53,409 - __main__ - WARNING - Attempt 16: Please wait for sglang server to become ready... 2025-08-24 23:59:54,483 - __main__ - WARNING - Attempt 17: Please wait for sglang server to become ready... 2025-08-24 23:59:55,552 - __main__ - WARNING - Attempt 18: Please wait for sglang server to become ready... 2025-08-24 23:59:56,603 - __main__ - WARNING - Attempt 19: Please wait for sglang server to become ready... 2025-08-24 23:59:56,899 - sglang - INFO - [2025-08-24 23:59:56 TP0] Load weight begin. avail mem=23.33 GB 2025-08-24 23:59:56,900 - __main__ - INFO - [2025-08-24 23:59:56 TP0] Load weight begin. avail mem=23.33 GB 2025-08-24 23:59:57,458 - sglang - INFO - Loading safetensors checkpoint shards: 0% Completed | 0/4 [00:00 - Response exceeded model_max_context, cannot use this response 2025-08-25 00:01:47,389 - __main__ - INFO - Built page query for ./workspace/UNETR.pdf-5 2025-08-25 00:01:47,611 - sglang - INFO - [2025-08-25 00:01:47 TP0] Prefill batch. #new-seq: 1, #new-token: 3206, #cached-token: 0, cache hit rate: 0.00%, token usage: 0.15, #running-req: 1, #queue-req: 0 2025-08-25 00:01:47,611 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:01:48,704 - sglang - INFO - [2025-08-25 00:01:48 TP0] Decode batch. #running-req: 2, #token: 8838, token usage: 0.23, gen throughput (token/s): 30.33, #queue-req: 0 2025-08-25 00:01:48,704 - __main__ - INFO - sglang running req: 2 queue req: 0 2025-08-25 00:01:49,557 - sglang - INFO - [2025-08-25 00:01:49 TP0] Decode batch. #running-req: 2, #token: 8918, token usage: 0.23, gen throughput (token/s): 93.81, #queue-req: 0 2025-08-25 00:01:49,557 - __main__ - INFO - sglang running req: 2 queue req: 0 2025-08-25 00:01:50,409 - sglang - INFO - [2025-08-25 00:01:50 TP0] Decode batch. #running-req: 2, #token: 8998, token usage: 0.24, gen throughput (token/s): 93.85, #queue-req: 0 2025-08-25 00:01:50,410 - __main__ - INFO - sglang running req: 2 queue req: 0 2025-08-25 00:01:51,262 - sglang - INFO - [2025-08-25 00:01:51 TP0] Decode batch. #running-req: 2, #token: 9078, token usage: 0.24, gen throughput (token/s): 93.76, #queue-req: 0 2025-08-25 00:01:51,263 - __main__ - INFO - sglang running req: 2 queue req: 0 2025-08-25 00:01:52,116 - sglang - INFO - [2025-08-25 00:01:52 TP0] Decode batch. #running-req: 2, #token: 9158, token usage: 0.24, gen throughput (token/s): 93.73, #queue-req: 0 2025-08-25 00:01:52,116 - __main__ - INFO - sglang running req: 2 queue req: 0 2025-08-25 00:01:52,964 - sglang - INFO - [2025-08-25 00:01:52 TP0] Decode batch. #running-req: 1, #token: 3414, token usage: 0.09, gen throughput (token/s): 76.63, #queue-req: 0 2025-08-25 00:01:52,965 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:01:53,802 - sglang - INFO - [2025-08-25 00:01:53 TP0] Decode batch. #running-req: 1, #token: 3454, token usage: 0.09, gen throughput (token/s): 47.75, #queue-req: 0 2025-08-25 00:01:53,802 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:01:54,166 - __main__ - INFO - Queue remaining: 0 2025-08-25 00:01:54,166 - __main__ - INFO - Metric Name Lifetime (tokens/sec) Recently (tokens/sec) ---------------------------------------------------------------------------------- sglang_input_tokens 260.08 260.08 sglang_output_tokens 79.05 79.05 2025-08-25 00:01:54,166 - __main__ - INFO - Worker ID | finished | started ----------+----------+-------- 0 | 10 | 11 2025-08-25 00:01:54,639 - sglang - INFO - [2025-08-25 00:01:54 TP0] Decode batch. #running-req: 1, #token: 3494, token usage: 0.09, gen throughput (token/s): 47.78, #queue-req: 0 2025-08-25 00:01:54,639 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:01:55,477 - sglang - INFO - [2025-08-25 00:01:55 TP0] Decode batch. #running-req: 1, #token: 3534, token usage: 0.09, gen throughput (token/s): 47.75, #queue-req: 0 2025-08-25 00:01:55,477 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:01:56,316 - sglang - INFO - [2025-08-25 00:01:56 TP0] Decode batch. #running-req: 1, #token: 3574, token usage: 0.09, gen throughput (token/s): 47.69, #queue-req: 0 2025-08-25 00:01:56,316 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:01:57,154 - sglang - INFO - [2025-08-25 00:01:57 TP0] Decode batch. #running-req: 1, #token: 3614, token usage: 0.10, gen throughput (token/s): 47.71, #queue-req: 0 2025-08-25 00:01:57,154 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:01:57,992 - sglang - INFO - [2025-08-25 00:01:57 TP0] Decode batch. #running-req: 1, #token: 3654, token usage: 0.10, gen throughput (token/s): 47.71, #queue-req: 0 2025-08-25 00:01:57,993 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:01:58,832 - sglang - INFO - [2025-08-25 00:01:58 TP0] Decode batch. #running-req: 1, #token: 3694, token usage: 0.10, gen throughput (token/s): 47.64, #queue-req: 0 2025-08-25 00:01:58,832 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:01:59,671 - sglang - INFO - [2025-08-25 00:01:59 TP0] Decode batch. #running-req: 1, #token: 3734, token usage: 0.10, gen throughput (token/s): 47.66, #queue-req: 0 2025-08-25 00:01:59,671 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:02:00,511 - sglang - INFO - [2025-08-25 00:02:00 TP0] Decode batch. #running-req: 1, #token: 3774, token usage: 0.10, gen throughput (token/s): 47.62, #queue-req: 0 2025-08-25 00:02:00,512 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:02:01,351 - sglang - INFO - [2025-08-25 00:02:01 TP0] Decode batch. #running-req: 1, #token: 3814, token usage: 0.10, gen throughput (token/s): 47.64, #queue-req: 0 2025-08-25 00:02:01,351 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:02:02,190 - sglang - INFO - [2025-08-25 00:02:02 TP0] Decode batch. #running-req: 1, #token: 3854, token usage: 0.10, gen throughput (token/s): 47.65, #queue-req: 0 2025-08-25 00:02:02,191 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:02:03,030 - sglang - INFO - [2025-08-25 00:02:03 TP0] Decode batch. #running-req: 1, #token: 3894, token usage: 0.10, gen throughput (token/s): 47.67, #queue-req: 0 2025-08-25 00:02:03,030 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:02:03,869 - sglang - INFO - [2025-08-25 00:02:03 TP0] Decode batch. #running-req: 1, #token: 3934, token usage: 0.10, gen throughput (token/s): 47.62, #queue-req: 0 2025-08-25 00:02:03,870 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:02:04,167 - __main__ - INFO - Queue remaining: 0 2025-08-25 00:02:04,168 - __main__ - INFO - Metric Name Lifetime (tokens/sec) Recently (tokens/sec) ---------------------------------------------------------------------------------- sglang_input_tokens 242.38 242.38 sglang_output_tokens 73.67 73.67 2025-08-25 00:02:04,168 - __main__ - INFO - Worker ID | finished | started ----------+----------+-------- 0 | 10 | 11 2025-08-25 00:02:04,709 - sglang - INFO - [2025-08-25 00:02:04 TP0] Decode batch. #running-req: 1, #token: 3974, token usage: 0.10, gen throughput (token/s): 47.66, #queue-req: 0 2025-08-25 00:02:04,709 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:02:05,548 - sglang - INFO - [2025-08-25 00:02:05 TP0] Decode batch. #running-req: 1, #token: 4014, token usage: 0.11, gen throughput (token/s): 47.67, #queue-req: 0 2025-08-25 00:02:05,548 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:02:06,387 - sglang - INFO - [2025-08-25 00:02:06 TP0] Decode batch. #running-req: 1, #token: 4054, token usage: 0.11, gen throughput (token/s): 47.67, #queue-req: 0 2025-08-25 00:02:06,387 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:02:07,226 - sglang - INFO - [2025-08-25 00:02:07 TP0] Decode batch. #running-req: 1, #token: 4094, token usage: 0.11, gen throughput (token/s): 47.65, #queue-req: 0 2025-08-25 00:02:07,227 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:02:08,068 - sglang - INFO - [2025-08-25 00:02:08 TP0] Decode batch. #running-req: 1, #token: 4134, token usage: 0.11, gen throughput (token/s): 47.53, #queue-req: 0 2025-08-25 00:02:08,068 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:02:08,911 - sglang - INFO - [2025-08-25 00:02:08 TP0] Decode batch. #running-req: 1, #token: 4174, token usage: 0.11, gen throughput (token/s): 47.49, #queue-req: 0 2025-08-25 00:02:08,911 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:02:09,752 - sglang - INFO - [2025-08-25 00:02:09 TP0] Decode batch. #running-req: 1, #token: 4214, token usage: 0.11, gen throughput (token/s): 47.57, #queue-req: 0 2025-08-25 00:02:09,752 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:02:10,592 - sglang - INFO - [2025-08-25 00:02:10 TP0] Decode batch. #running-req: 1, #token: 4254, token usage: 0.11, gen throughput (token/s): 47.57, #queue-req: 0 2025-08-25 00:02:10,593 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:02:11,434 - sglang - INFO - [2025-08-25 00:02:11 TP0] Decode batch. #running-req: 1, #token: 4294, token usage: 0.11, gen throughput (token/s): 47.52, #queue-req: 0 2025-08-25 00:02:11,434 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:02:12,276 - sglang - INFO - [2025-08-25 00:02:12 TP0] Decode batch. #running-req: 1, #token: 4334, token usage: 0.11, gen throughput (token/s): 47.49, #queue-req: 0 2025-08-25 00:02:12,277 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:02:13,120 - sglang - INFO - [2025-08-25 00:02:13 TP0] Decode batch. #running-req: 1, #token: 4374, token usage: 0.12, gen throughput (token/s): 47.40, #queue-req: 0 2025-08-25 00:02:13,121 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:02:13,963 - sglang - INFO - [2025-08-25 00:02:13 TP0] Decode batch. #running-req: 1, #token: 4414, token usage: 0.12, gen throughput (token/s): 47.47, #queue-req: 0 2025-08-25 00:02:13,963 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:02:14,170 - __main__ - INFO - Queue remaining: 0 2025-08-25 00:02:14,170 - __main__ - INFO - Metric Name Lifetime (tokens/sec) Recently (tokens/sec) ---------------------------------------------------------------------------------- sglang_input_tokens 226.93 226.93 sglang_output_tokens 68.97 68.97 2025-08-25 00:02:14,170 - __main__ - INFO - Worker ID | finished | started ----------+----------+-------- 0 | 10 | 11 2025-08-25 00:02:14,804 - sglang - INFO - [2025-08-25 00:02:14 TP0] Decode batch. #running-req: 1, #token: 4454, token usage: 0.12, gen throughput (token/s): 47.53, #queue-req: 0 2025-08-25 00:02:14,805 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:02:15,646 - sglang - INFO - [2025-08-25 00:02:15 TP0] Decode batch. #running-req: 1, #token: 4494, token usage: 0.12, gen throughput (token/s): 47.53, #queue-req: 0 2025-08-25 00:02:15,646 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:02:16,489 - sglang - INFO - [2025-08-25 00:02:16 TP0] Decode batch. #running-req: 1, #token: 4534, token usage: 0.12, gen throughput (token/s): 47.47, #queue-req: 0 2025-08-25 00:02:16,489 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:02:17,331 - sglang - INFO - [2025-08-25 00:02:17 TP0] Decode batch. #running-req: 1, #token: 4574, token usage: 0.12, gen throughput (token/s): 47.52, #queue-req: 0 2025-08-25 00:02:17,331 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:02:18,173 - sglang - INFO - [2025-08-25 00:02:18 TP0] Decode batch. #running-req: 1, #token: 4614, token usage: 0.12, gen throughput (token/s): 47.47, #queue-req: 0 2025-08-25 00:02:18,173 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:02:19,016 - sglang - INFO - [2025-08-25 00:02:19 TP0] Decode batch. #running-req: 1, #token: 4654, token usage: 0.12, gen throughput (token/s): 47.45, #queue-req: 0 2025-08-25 00:02:19,017 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:02:19,859 - sglang - INFO - [2025-08-25 00:02:19 TP0] Decode batch. #running-req: 1, #token: 4694, token usage: 0.12, gen throughput (token/s): 47.44, #queue-req: 0 2025-08-25 00:02:19,860 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:02:20,704 - sglang - INFO - [2025-08-25 00:02:20 TP0] Decode batch. #running-req: 1, #token: 4734, token usage: 0.12, gen throughput (token/s): 47.37, #queue-req: 0 2025-08-25 00:02:20,704 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:02:21,548 - sglang - INFO - [2025-08-25 00:02:21 TP0] Decode batch. #running-req: 1, #token: 4774, token usage: 0.13, gen throughput (token/s): 47.40, #queue-req: 0 2025-08-25 00:02:21,548 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:02:22,392 - sglang - INFO - [2025-08-25 00:02:22 TP0] Decode batch. #running-req: 1, #token: 4814, token usage: 0.13, gen throughput (token/s): 47.37, #queue-req: 0 2025-08-25 00:02:22,392 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:02:23,236 - sglang - INFO - [2025-08-25 00:02:23 TP0] Decode batch. #running-req: 1, #token: 4854, token usage: 0.13, gen throughput (token/s): 47.40, #queue-req: 0 2025-08-25 00:02:23,236 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:02:24,089 - sglang - INFO - [2025-08-25 00:02:24 TP0] Decode batch. #running-req: 1, #token: 4894, token usage: 0.13, gen throughput (token/s): 46.90, #queue-req: 0 2025-08-25 00:02:24,089 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:02:24,171 - __main__ - INFO - Queue remaining: 0 2025-08-25 00:02:24,172 - __main__ - INFO - Metric Name Lifetime (tokens/sec) Recently (tokens/sec) ---------------------------------------------------------------------------------- sglang_input_tokens 213.34 213.34 sglang_output_tokens 64.84 64.84 2025-08-25 00:02:24,172 - __main__ - INFO - Worker ID | finished | started ----------+----------+-------- 0 | 10 | 11 2025-08-25 00:02:24,939 - sglang - INFO - [2025-08-25 00:02:24 TP0] Decode batch. #running-req: 1, #token: 4934, token usage: 0.13, gen throughput (token/s): 47.04, #queue-req: 0 2025-08-25 00:02:24,939 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:02:25,790 - sglang - INFO - [2025-08-25 00:02:25 TP0] Decode batch. #running-req: 1, #token: 4974, token usage: 0.13, gen throughput (token/s): 46.99, #queue-req: 0 2025-08-25 00:02:25,791 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:02:26,640 - sglang - INFO - [2025-08-25 00:02:26 TP0] Decode batch. #running-req: 1, #token: 5014, token usage: 0.13, gen throughput (token/s): 47.09, #queue-req: 0 2025-08-25 00:02:26,640 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:02:27,485 - sglang - INFO - [2025-08-25 00:02:27 TP0] Decode batch. #running-req: 1, #token: 5054, token usage: 0.13, gen throughput (token/s): 47.35, #queue-req: 0 2025-08-25 00:02:27,485 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:02:28,330 - sglang - INFO - [2025-08-25 00:02:28 TP0] Decode batch. #running-req: 1, #token: 5094, token usage: 0.13, gen throughput (token/s): 47.30, #queue-req: 0 2025-08-25 00:02:28,331 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:02:29,176 - sglang - INFO - [2025-08-25 00:02:29 TP0] Decode batch. #running-req: 1, #token: 5134, token usage: 0.14, gen throughput (token/s): 47.29, #queue-req: 0 2025-08-25 00:02:29,177 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:02:30,023 - sglang - INFO - [2025-08-25 00:02:30 TP0] Decode batch. #running-req: 1, #token: 5174, token usage: 0.14, gen throughput (token/s): 47.23, #queue-req: 0 2025-08-25 00:02:30,024 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:02:30,871 - sglang - INFO - [2025-08-25 00:02:30 TP0] Decode batch. #running-req: 1, #token: 5214, token usage: 0.14, gen throughput (token/s): 47.21, #queue-req: 0 2025-08-25 00:02:30,871 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:02:31,716 - sglang - INFO - [2025-08-25 00:02:31 TP0] Decode batch. #running-req: 1, #token: 5254, token usage: 0.14, gen throughput (token/s): 47.30, #queue-req: 0 2025-08-25 00:02:31,717 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:02:32,564 - sglang - INFO - [2025-08-25 00:02:32 TP0] Decode batch. #running-req: 1, #token: 5294, token usage: 0.14, gen throughput (token/s): 47.22, #queue-req: 0 2025-08-25 00:02:32,564 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:02:33,412 - sglang - INFO - [2025-08-25 00:02:33 TP0] Decode batch. #running-req: 1, #token: 5334, token usage: 0.14, gen throughput (token/s): 47.16, #queue-req: 0 2025-08-25 00:02:33,412 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:02:34,172 - __main__ - INFO - Queue remaining: 0 2025-08-25 00:02:34,173 - __main__ - INFO - Metric Name Lifetime (tokens/sec) Recently (tokens/sec) ---------------------------------------------------------------------------------- sglang_input_tokens 201.28 201.28 sglang_output_tokens 61.18 61.18 2025-08-25 00:02:34,173 - __main__ - INFO - Worker ID | finished | started ----------+----------+-------- 0 | 10 | 11 2025-08-25 00:02:34,260 - sglang - INFO - [2025-08-25 00:02:34 TP0] Decode batch. #running-req: 1, #token: 5374, token usage: 0.14, gen throughput (token/s): 47.17, #queue-req: 0 2025-08-25 00:02:34,260 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:02:35,108 - sglang - INFO - [2025-08-25 00:02:35 TP0] Decode batch. #running-req: 1, #token: 5414, token usage: 0.14, gen throughput (token/s): 47.15, #queue-req: 0 2025-08-25 00:02:35,108 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:02:35,955 - sglang - INFO - [2025-08-25 00:02:35 TP0] Decode batch. #running-req: 1, #token: 5454, token usage: 0.14, gen throughput (token/s): 47.22, #queue-req: 0 2025-08-25 00:02:35,955 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:02:36,802 - sglang - INFO - [2025-08-25 00:02:36 TP0] Decode batch. #running-req: 1, #token: 5494, token usage: 0.14, gen throughput (token/s): 47.21, #queue-req: 0 2025-08-25 00:02:36,802 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:02:37,649 - sglang - INFO - [2025-08-25 00:02:37 TP0] Decode batch. #running-req: 1, #token: 5534, token usage: 0.15, gen throughput (token/s): 47.21, #queue-req: 0 2025-08-25 00:02:37,650 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:02:38,497 - sglang - INFO - [2025-08-25 00:02:38 TP0] Decode batch. #running-req: 1, #token: 5574, token usage: 0.15, gen throughput (token/s): 47.17, #queue-req: 0 2025-08-25 00:02:38,498 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:02:39,347 - sglang - INFO - [2025-08-25 00:02:39 TP0] Decode batch. #running-req: 1, #token: 5614, token usage: 0.15, gen throughput (token/s): 47.09, #queue-req: 0 2025-08-25 00:02:39,347 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:02:40,195 - sglang - INFO - [2025-08-25 00:02:40 TP0] Decode batch. #running-req: 1, #token: 5654, token usage: 0.15, gen throughput (token/s): 47.16, #queue-req: 0 2025-08-25 00:02:40,195 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:02:41,043 - sglang - INFO - [2025-08-25 00:02:41 TP0] Decode batch. #running-req: 1, #token: 5694, token usage: 0.15, gen throughput (token/s): 47.16, #queue-req: 0 2025-08-25 00:02:41,044 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:02:41,891 - sglang - INFO - [2025-08-25 00:02:41 TP0] Decode batch. #running-req: 1, #token: 5734, token usage: 0.15, gen throughput (token/s): 47.18, #queue-req: 0 2025-08-25 00:02:41,891 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:02:42,740 - sglang - INFO - [2025-08-25 00:02:42 TP0] Decode batch. #running-req: 1, #token: 5774, token usage: 0.15, gen throughput (token/s): 47.15, #queue-req: 0 2025-08-25 00:02:42,740 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:02:43,590 - sglang - INFO - [2025-08-25 00:02:43 TP0] Decode batch. #running-req: 1, #token: 5814, token usage: 0.15, gen throughput (token/s): 47.06, #queue-req: 0 2025-08-25 00:02:43,590 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:02:44,174 - __main__ - INFO - Queue remaining: 0 2025-08-25 00:02:44,174 - __main__ - INFO - Metric Name Lifetime (tokens/sec) Recently (tokens/sec) ---------------------------------------------------------------------------------- sglang_input_tokens 190.51 190.51 sglang_output_tokens 57.91 57.91 2025-08-25 00:02:44,174 - __main__ - INFO - Worker ID | finished | started ----------+----------+-------- 0 | 10 | 11 2025-08-25 00:02:44,439 - sglang - INFO - [2025-08-25 00:02:44 TP0] Decode batch. #running-req: 1, #token: 5854, token usage: 0.15, gen throughput (token/s): 47.12, #queue-req: 0 2025-08-25 00:02:44,439 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:02:45,287 - sglang - INFO - [2025-08-25 00:02:45 TP0] Decode batch. #running-req: 1, #token: 5894, token usage: 0.16, gen throughput (token/s): 47.17, #queue-req: 0 2025-08-25 00:02:45,287 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:02:46,133 - sglang - INFO - [2025-08-25 00:02:46 TP0] Decode batch. #running-req: 1, #token: 5934, token usage: 0.16, gen throughput (token/s): 47.24, #queue-req: 0 2025-08-25 00:02:46,133 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:02:46,982 - sglang - INFO - [2025-08-25 00:02:46 TP0] Decode batch. #running-req: 1, #token: 5974, token usage: 0.16, gen throughput (token/s): 47.13, #queue-req: 0 2025-08-25 00:02:46,982 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:02:47,830 - sglang - INFO - [2025-08-25 00:02:47 TP0] Decode batch. #running-req: 1, #token: 6014, token usage: 0.16, gen throughput (token/s): 47.17, #queue-req: 0 2025-08-25 00:02:47,830 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:02:48,676 - sglang - INFO - [2025-08-25 00:02:48 TP0] Decode batch. #running-req: 1, #token: 6054, token usage: 0.16, gen throughput (token/s): 47.25, #queue-req: 0 2025-08-25 00:02:48,677 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:02:49,525 - sglang - INFO - [2025-08-25 00:02:49 TP0] Decode batch. #running-req: 1, #token: 6094, token usage: 0.16, gen throughput (token/s): 47.16, #queue-req: 0 2025-08-25 00:02:49,525 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:02:50,373 - sglang - INFO - [2025-08-25 00:02:50 TP0] Decode batch. #running-req: 1, #token: 6134, token usage: 0.16, gen throughput (token/s): 47.18, #queue-req: 0 2025-08-25 00:02:50,373 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:02:51,221 - sglang - INFO - [2025-08-25 00:02:51 TP0] Decode batch. #running-req: 1, #token: 6174, token usage: 0.16, gen throughput (token/s): 47.16, #queue-req: 0 2025-08-25 00:02:51,221 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:02:51,885 - __main__ - WARNING - JSON decode error on attempt 1 for ./workspace/UNETR.pdf-5: Unterminated string starting at: line 1 column 126 (char 125) 2025-08-25 00:02:52,151 - __main__ - INFO - Built page query for ./workspace/UNETR.pdf-5 2025-08-25 00:02:52,318 - sglang - INFO - [2025-08-25 00:02:52 TP0] Prefill batch. #new-seq: 1, #new-token: 3125, #cached-token: 0, cache hit rate: 0.00%, token usage: 0.00, #running-req: 0, #queue-req: 0 2025-08-25 00:02:52,318 - __main__ - INFO - sglang running req: 0 queue req: 0 2025-08-25 00:02:53,418 - sglang - INFO - [2025-08-25 00:02:53 TP0] Decode batch. #running-req: 1, #token: 3134, token usage: 0.08, gen throughput (token/s): 18.21, #queue-req: 0 2025-08-25 00:02:53,418 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:02:54,175 - __main__ - INFO - Queue remaining: 0 2025-08-25 00:02:54,175 - __main__ - INFO - Metric Name Lifetime (tokens/sec) Recently (tokens/sec) ---------------------------------------------------------------------------------- sglang_input_tokens 197.11 197.11 sglang_output_tokens 70.20 70.20 2025-08-25 00:02:54,175 - __main__ - INFO - Worker ID | finished | started ----------+----------+-------- 0 | 10 | 11 2025-08-25 00:02:54,256 - sglang - INFO - [2025-08-25 00:02:54 TP0] Decode batch. #running-req: 1, #token: 3174, token usage: 0.08, gen throughput (token/s): 47.70, #queue-req: 0 2025-08-25 00:02:54,257 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:02:55,095 - sglang - INFO - [2025-08-25 00:02:55 TP0] Decode batch. #running-req: 1, #token: 3214, token usage: 0.08, gen throughput (token/s): 47.71, #queue-req: 0 2025-08-25 00:02:55,095 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:02:55,934 - sglang - INFO - [2025-08-25 00:02:55 TP0] Decode batch. #running-req: 1, #token: 3254, token usage: 0.09, gen throughput (token/s): 47.65, #queue-req: 0 2025-08-25 00:02:55,935 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:02:56,774 - sglang - INFO - [2025-08-25 00:02:56 TP0] Decode batch. #running-req: 1, #token: 3294, token usage: 0.09, gen throughput (token/s): 47.66, #queue-req: 0 2025-08-25 00:02:56,774 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:02:57,613 - sglang - INFO - [2025-08-25 00:02:57 TP0] Decode batch. #running-req: 1, #token: 3334, token usage: 0.09, gen throughput (token/s): 47.65, #queue-req: 0 2025-08-25 00:02:57,614 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:02:58,453 - sglang - INFO - [2025-08-25 00:02:58 TP0] Decode batch. #running-req: 1, #token: 3374, token usage: 0.09, gen throughput (token/s): 47.63, #queue-req: 0 2025-08-25 00:02:58,453 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:02:59,292 - sglang - INFO - [2025-08-25 00:02:59 TP0] Decode batch. #running-req: 1, #token: 3414, token usage: 0.09, gen throughput (token/s): 47.71, #queue-req: 0 2025-08-25 00:02:59,292 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:03:00,130 - sglang - INFO - [2025-08-25 00:03:00 TP0] Decode batch. #running-req: 1, #token: 3454, token usage: 0.09, gen throughput (token/s): 47.69, #queue-req: 0 2025-08-25 00:03:00,131 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:03:00,970 - sglang - INFO - [2025-08-25 00:03:00 TP0] Decode batch. #running-req: 1, #token: 3494, token usage: 0.09, gen throughput (token/s): 47.62, #queue-req: 0 2025-08-25 00:03:00,971 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:03:01,810 - sglang - INFO - [2025-08-25 00:03:01 TP0] Decode batch. #running-req: 1, #token: 3534, token usage: 0.09, gen throughput (token/s): 47.65, #queue-req: 0 2025-08-25 00:03:01,810 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:03:02,649 - sglang - INFO - [2025-08-25 00:03:02 TP0] Decode batch. #running-req: 1, #token: 3574, token usage: 0.09, gen throughput (token/s): 47.69, #queue-req: 0 2025-08-25 00:03:02,649 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:03:03,488 - sglang - INFO - [2025-08-25 00:03:03 TP0] Decode batch. #running-req: 1, #token: 3614, token usage: 0.10, gen throughput (token/s): 47.67, #queue-req: 0 2025-08-25 00:03:03,488 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:03:04,176 - __main__ - INFO - Queue remaining: 0 2025-08-25 00:03:04,177 - __main__ - INFO - Metric Name Lifetime (tokens/sec) Recently (tokens/sec) ---------------------------------------------------------------------------------- sglang_input_tokens 187.59 187.59 sglang_output_tokens 66.80 66.80 2025-08-25 00:03:04,177 - __main__ - INFO - Worker ID | finished | started ----------+----------+-------- 0 | 10 | 11 2025-08-25 00:03:04,328 - sglang - INFO - [2025-08-25 00:03:04 TP0] Decode batch. #running-req: 1, #token: 3654, token usage: 0.10, gen throughput (token/s): 47.59, #queue-req: 0 2025-08-25 00:03:04,329 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:03:05,168 - sglang - INFO - [2025-08-25 00:03:05 TP0] Decode batch. #running-req: 1, #token: 3694, token usage: 0.10, gen throughput (token/s): 47.64, #queue-req: 0 2025-08-25 00:03:05,168 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:03:06,007 - sglang - INFO - [2025-08-25 00:03:06 TP0] Decode batch. #running-req: 1, #token: 3734, token usage: 0.10, gen throughput (token/s): 47.68, #queue-req: 0 2025-08-25 00:03:06,007 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:03:06,845 - sglang - INFO - [2025-08-25 00:03:06 TP0] Decode batch. #running-req: 1, #token: 3774, token usage: 0.10, gen throughput (token/s): 47.71, #queue-req: 0 2025-08-25 00:03:06,845 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:03:07,685 - sglang - INFO - [2025-08-25 00:03:07 TP0] Decode batch. #running-req: 1, #token: 3814, token usage: 0.10, gen throughput (token/s): 47.65, #queue-req: 0 2025-08-25 00:03:07,685 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:03:08,524 - sglang - INFO - [2025-08-25 00:03:08 TP0] Decode batch. #running-req: 1, #token: 3854, token usage: 0.10, gen throughput (token/s): 47.65, #queue-req: 0 2025-08-25 00:03:08,524 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:03:09,363 - sglang - INFO - [2025-08-25 00:03:09 TP0] Decode batch. #running-req: 1, #token: 3894, token usage: 0.10, gen throughput (token/s): 47.66, #queue-req: 0 2025-08-25 00:03:09,364 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:03:10,204 - sglang - INFO - [2025-08-25 00:03:10 TP0] Decode batch. #running-req: 1, #token: 3934, token usage: 0.10, gen throughput (token/s): 47.61, #queue-req: 0 2025-08-25 00:03:10,204 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:03:11,043 - sglang - INFO - [2025-08-25 00:03:11 TP0] Decode batch. #running-req: 1, #token: 3974, token usage: 0.10, gen throughput (token/s): 47.66, #queue-req: 0 2025-08-25 00:03:11,043 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:03:11,884 - sglang - INFO - [2025-08-25 00:03:11 TP0] Decode batch. #running-req: 1, #token: 4014, token usage: 0.11, gen throughput (token/s): 47.56, #queue-req: 0 2025-08-25 00:03:11,884 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:03:12,724 - sglang - INFO - [2025-08-25 00:03:12 TP0] Decode batch. #running-req: 1, #token: 4054, token usage: 0.11, gen throughput (token/s): 47.59, #queue-req: 0 2025-08-25 00:03:12,724 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:03:13,565 - sglang - INFO - [2025-08-25 00:03:13 TP0] Decode batch. #running-req: 1, #token: 4094, token usage: 0.11, gen throughput (token/s): 47.57, #queue-req: 0 2025-08-25 00:03:13,565 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:03:14,178 - __main__ - INFO - Queue remaining: 0 2025-08-25 00:03:14,178 - __main__ - INFO - Metric Name Lifetime (tokens/sec) Recently (tokens/sec) ---------------------------------------------------------------------------------- sglang_input_tokens 178.94 178.94 sglang_output_tokens 63.72 63.72 2025-08-25 00:03:14,178 - __main__ - INFO - Worker ID | finished | started ----------+----------+-------- 0 | 10 | 11 2025-08-25 00:03:14,405 - sglang - INFO - [2025-08-25 00:03:14 TP0] Decode batch. #running-req: 1, #token: 4134, token usage: 0.11, gen throughput (token/s): 47.60, #queue-req: 0 2025-08-25 00:03:14,406 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:03:15,246 - sglang - INFO - [2025-08-25 00:03:15 TP0] Decode batch. #running-req: 1, #token: 4174, token usage: 0.11, gen throughput (token/s): 47.59, #queue-req: 0 2025-08-25 00:03:15,246 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:03:16,087 - sglang - INFO - [2025-08-25 00:03:16 TP0] Decode batch. #running-req: 1, #token: 4214, token usage: 0.11, gen throughput (token/s): 47.55, #queue-req: 0 2025-08-25 00:03:16,087 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:03:16,927 - sglang - INFO - [2025-08-25 00:03:16 TP0] Decode batch. #running-req: 1, #token: 4254, token usage: 0.11, gen throughput (token/s): 47.60, #queue-req: 0 2025-08-25 00:03:16,928 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:03:17,768 - sglang - INFO - [2025-08-25 00:03:17 TP0] Decode batch. #running-req: 1, #token: 4294, token usage: 0.11, gen throughput (token/s): 47.57, #queue-req: 0 2025-08-25 00:03:17,769 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:03:18,610 - sglang - INFO - [2025-08-25 00:03:18 TP0] Decode batch. #running-req: 1, #token: 4334, token usage: 0.11, gen throughput (token/s): 47.52, #queue-req: 0 2025-08-25 00:03:18,610 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:03:19,453 - sglang - INFO - [2025-08-25 00:03:19 TP0] Decode batch. #running-req: 1, #token: 4374, token usage: 0.12, gen throughput (token/s): 47.47, #queue-req: 0 2025-08-25 00:03:19,453 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:03:20,294 - sglang - INFO - [2025-08-25 00:03:20 TP0] Decode batch. #running-req: 1, #token: 4414, token usage: 0.12, gen throughput (token/s): 47.52, #queue-req: 0 2025-08-25 00:03:20,295 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:03:21,136 - sglang - INFO - [2025-08-25 00:03:21 TP0] Decode batch. #running-req: 1, #token: 4454, token usage: 0.12, gen throughput (token/s): 47.52, #queue-req: 0 2025-08-25 00:03:21,136 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:03:21,978 - sglang - INFO - [2025-08-25 00:03:21 TP0] Decode batch. #running-req: 1, #token: 4494, token usage: 0.12, gen throughput (token/s): 47.51, #queue-req: 0 2025-08-25 00:03:21,978 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:03:22,821 - sglang - INFO - [2025-08-25 00:03:22 TP0] Decode batch. #running-req: 1, #token: 4534, token usage: 0.12, gen throughput (token/s): 47.46, #queue-req: 0 2025-08-25 00:03:22,821 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:03:23,664 - sglang - INFO - [2025-08-25 00:03:23 TP0] Decode batch. #running-req: 1, #token: 4574, token usage: 0.12, gen throughput (token/s): 47.46, #queue-req: 0 2025-08-25 00:03:23,664 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:03:24,179 - __main__ - INFO - Queue remaining: 0 2025-08-25 00:03:24,180 - __main__ - INFO - Metric Name Lifetime (tokens/sec) Recently (tokens/sec) ---------------------------------------------------------------------------------- sglang_input_tokens 171.06 171.06 sglang_output_tokens 60.92 60.92 2025-08-25 00:03:24,180 - __main__ - INFO - Worker ID | finished | started ----------+----------+-------- 0 | 10 | 11 2025-08-25 00:03:24,507 - sglang - INFO - [2025-08-25 00:03:24 TP0] Decode batch. #running-req: 1, #token: 4614, token usage: 0.12, gen throughput (token/s): 47.42, #queue-req: 0 2025-08-25 00:03:24,508 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:03:25,350 - sglang - INFO - [2025-08-25 00:03:25 TP0] Decode batch. #running-req: 1, #token: 4654, token usage: 0.12, gen throughput (token/s): 47.44, #queue-req: 0 2025-08-25 00:03:25,351 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:03:26,193 - sglang - INFO - [2025-08-25 00:03:26 TP0] Decode batch. #running-req: 1, #token: 4694, token usage: 0.12, gen throughput (token/s): 47.46, #queue-req: 0 2025-08-25 00:03:26,194 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:03:27,037 - sglang - INFO - [2025-08-25 00:03:27 TP0] Decode batch. #running-req: 1, #token: 4734, token usage: 0.12, gen throughput (token/s): 47.40, #queue-req: 0 2025-08-25 00:03:27,037 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:03:27,881 - sglang - INFO - [2025-08-25 00:03:27 TP0] Decode batch. #running-req: 1, #token: 4774, token usage: 0.13, gen throughput (token/s): 47.43, #queue-req: 0 2025-08-25 00:03:27,881 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:03:28,723 - sglang - INFO - [2025-08-25 00:03:28 TP0] Decode batch. #running-req: 1, #token: 4814, token usage: 0.13, gen throughput (token/s): 47.48, #queue-req: 0 2025-08-25 00:03:28,723 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:03:29,565 - sglang - INFO - [2025-08-25 00:03:29 TP0] Decode batch. #running-req: 1, #token: 4854, token usage: 0.13, gen throughput (token/s): 47.50, #queue-req: 0 2025-08-25 00:03:29,565 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:03:30,408 - sglang - INFO - [2025-08-25 00:03:30 TP0] Decode batch. #running-req: 1, #token: 4894, token usage: 0.13, gen throughput (token/s): 47.45, #queue-req: 0 2025-08-25 00:03:30,408 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:03:31,254 - sglang - INFO - [2025-08-25 00:03:31 TP0] Decode batch. #running-req: 1, #token: 4934, token usage: 0.13, gen throughput (token/s): 47.31, #queue-req: 0 2025-08-25 00:03:31,254 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:03:32,098 - sglang - INFO - [2025-08-25 00:03:32 TP0] Decode batch. #running-req: 1, #token: 4974, token usage: 0.13, gen throughput (token/s): 47.35, #queue-req: 0 2025-08-25 00:03:32,098 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:03:32,943 - sglang - INFO - [2025-08-25 00:03:32 TP0] Decode batch. #running-req: 1, #token: 5014, token usage: 0.13, gen throughput (token/s): 47.33, #queue-req: 0 2025-08-25 00:03:32,944 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:03:33,787 - sglang - INFO - [2025-08-25 00:03:33 TP0] Decode batch. #running-req: 1, #token: 5054, token usage: 0.13, gen throughput (token/s): 47.39, #queue-req: 0 2025-08-25 00:03:33,788 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:03:34,181 - __main__ - INFO - Queue remaining: 0 2025-08-25 00:03:34,181 - __main__ - INFO - Metric Name Lifetime (tokens/sec) Recently (tokens/sec) ---------------------------------------------------------------------------------- sglang_input_tokens 163.84 163.84 sglang_output_tokens 58.35 58.35 2025-08-25 00:03:34,181 - __main__ - INFO - Worker ID | finished | started ----------+----------+-------- 0 | 10 | 11 2025-08-25 00:03:34,632 - sglang - INFO - [2025-08-25 00:03:34 TP0] Decode batch. #running-req: 1, #token: 5094, token usage: 0.13, gen throughput (token/s): 47.38, #queue-req: 0 2025-08-25 00:03:34,632 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:03:35,477 - sglang - INFO - [2025-08-25 00:03:35 TP0] Decode batch. #running-req: 1, #token: 5134, token usage: 0.14, gen throughput (token/s): 47.33, #queue-req: 0 2025-08-25 00:03:35,477 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:03:36,321 - sglang - INFO - [2025-08-25 00:03:36 TP0] Decode batch. #running-req: 1, #token: 5174, token usage: 0.14, gen throughput (token/s): 47.38, #queue-req: 0 2025-08-25 00:03:36,321 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:03:37,165 - sglang - INFO - [2025-08-25 00:03:37 TP0] Decode batch. #running-req: 1, #token: 5214, token usage: 0.14, gen throughput (token/s): 47.42, #queue-req: 0 2025-08-25 00:03:37,165 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:03:38,009 - sglang - INFO - [2025-08-25 00:03:38 TP0] Decode batch. #running-req: 1, #token: 5254, token usage: 0.14, gen throughput (token/s): 47.35, #queue-req: 0 2025-08-25 00:03:38,010 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:03:38,856 - sglang - INFO - [2025-08-25 00:03:38 TP0] Decode batch. #running-req: 1, #token: 5294, token usage: 0.14, gen throughput (token/s): 47.23, #queue-req: 0 2025-08-25 00:03:38,857 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:03:39,704 - sglang - INFO - [2025-08-25 00:03:39 TP0] Decode batch. #running-req: 1, #token: 5334, token usage: 0.14, gen throughput (token/s): 47.20, #queue-req: 0 2025-08-25 00:03:39,704 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:03:40,550 - sglang - INFO - [2025-08-25 00:03:40 TP0] Decode batch. #running-req: 1, #token: 5374, token usage: 0.14, gen throughput (token/s): 47.25, #queue-req: 0 2025-08-25 00:03:40,551 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:03:41,397 - sglang - INFO - [2025-08-25 00:03:41 TP0] Decode batch. #running-req: 1, #token: 5414, token usage: 0.14, gen throughput (token/s): 47.27, #queue-req: 0 2025-08-25 00:03:41,397 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:03:42,243 - sglang - INFO - [2025-08-25 00:03:42 TP0] Decode batch. #running-req: 1, #token: 5454, token usage: 0.14, gen throughput (token/s): 47.27, #queue-req: 0 2025-08-25 00:03:42,243 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:03:43,091 - sglang - INFO - [2025-08-25 00:03:43 TP0] Decode batch. #running-req: 1, #token: 5494, token usage: 0.14, gen throughput (token/s): 47.19, #queue-req: 0 2025-08-25 00:03:43,091 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:03:43,939 - sglang - INFO - [2025-08-25 00:03:43 TP0] Decode batch. #running-req: 1, #token: 5534, token usage: 0.15, gen throughput (token/s): 47.17, #queue-req: 0 2025-08-25 00:03:43,939 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:03:44,182 - __main__ - INFO - Queue remaining: 0 2025-08-25 00:03:44,183 - __main__ - INFO - Metric Name Lifetime (tokens/sec) Recently (tokens/sec) ---------------------------------------------------------------------------------- sglang_input_tokens 157.20 157.20 sglang_output_tokens 55.98 55.98 2025-08-25 00:03:44,183 - __main__ - INFO - Worker ID | finished | started ----------+----------+-------- 0 | 10 | 11 2025-08-25 00:03:44,786 - sglang - INFO - [2025-08-25 00:03:44 TP0] Decode batch. #running-req: 1, #token: 5574, token usage: 0.15, gen throughput (token/s): 47.22, #queue-req: 0 2025-08-25 00:03:44,786 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:03:45,632 - sglang - INFO - [2025-08-25 00:03:45 TP0] Decode batch. #running-req: 1, #token: 5614, token usage: 0.15, gen throughput (token/s): 47.29, #queue-req: 0 2025-08-25 00:03:45,632 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:03:46,478 - sglang - INFO - [2025-08-25 00:03:46 TP0] Decode batch. #running-req: 1, #token: 5654, token usage: 0.15, gen throughput (token/s): 47.26, #queue-req: 0 2025-08-25 00:03:46,478 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:03:47,326 - sglang - INFO - [2025-08-25 00:03:47 TP0] Decode batch. #running-req: 1, #token: 5694, token usage: 0.15, gen throughput (token/s): 47.17, #queue-req: 0 2025-08-25 00:03:47,326 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:03:48,173 - sglang - INFO - [2025-08-25 00:03:48 TP0] Decode batch. #running-req: 1, #token: 5734, token usage: 0.15, gen throughput (token/s): 47.19, #queue-req: 0 2025-08-25 00:03:48,174 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:03:49,021 - sglang - INFO - [2025-08-25 00:03:49 TP0] Decode batch. #running-req: 1, #token: 5774, token usage: 0.15, gen throughput (token/s): 47.17, #queue-req: 0 2025-08-25 00:03:49,022 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:03:49,867 - sglang - INFO - [2025-08-25 00:03:49 TP0] Decode batch. #running-req: 1, #token: 5814, token usage: 0.15, gen throughput (token/s): 47.28, #queue-req: 0 2025-08-25 00:03:49,868 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:03:50,714 - sglang - INFO - [2025-08-25 00:03:50 TP0] Decode batch. #running-req: 1, #token: 5854, token usage: 0.15, gen throughput (token/s): 47.25, #queue-req: 0 2025-08-25 00:03:50,714 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:03:51,562 - sglang - INFO - [2025-08-25 00:03:51 TP0] Decode batch. #running-req: 1, #token: 5894, token usage: 0.16, gen throughput (token/s): 47.18, #queue-req: 0 2025-08-25 00:03:51,562 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:03:52,410 - sglang - INFO - [2025-08-25 00:03:52 TP0] Decode batch. #running-req: 1, #token: 5934, token usage: 0.16, gen throughput (token/s): 47.16, #queue-req: 0 2025-08-25 00:03:52,410 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:03:53,257 - sglang - INFO - [2025-08-25 00:03:53 TP0] Decode batch. #running-req: 1, #token: 5974, token usage: 0.16, gen throughput (token/s): 47.20, #queue-req: 0 2025-08-25 00:03:53,258 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:03:54,104 - sglang - INFO - [2025-08-25 00:03:54 TP0] Decode batch. #running-req: 1, #token: 6014, token usage: 0.16, gen throughput (token/s): 47.24, #queue-req: 0 2025-08-25 00:03:54,104 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:03:54,184 - __main__ - INFO - Queue remaining: 0 2025-08-25 00:03:54,184 - __main__ - INFO - Metric Name Lifetime (tokens/sec) Recently (tokens/sec) ---------------------------------------------------------------------------------- sglang_input_tokens 151.08 151.08 sglang_output_tokens 53.80 53.80 2025-08-25 00:03:54,184 - __main__ - INFO - Worker ID | finished | started ----------+----------+-------- 0 | 10 | 11 2025-08-25 00:03:54,952 - sglang - INFO - [2025-08-25 00:03:54 TP0] Decode batch. #running-req: 1, #token: 6054, token usage: 0.16, gen throughput (token/s): 47.19, #queue-req: 0 2025-08-25 00:03:54,952 - __main__ - INFO - sglang running req: 1 queue req: 0 2025-08-25 00:03:55,277 - __main__ - INFO - Finished TaskGroup for worker on 73c9399482ed5cf37e1888c000e49ef82a30c10d 2025-08-25 00:03:55,278 - __main__ - INFO - Got 1 docs for 73c9399482ed5cf37e1888c000e49ef82a30c10d 2025-08-25 00:03:55,280 - __main__ - INFO - Worker 0 exiting due to empty queue 2025-08-25 00:03:55,280 - __main__ - INFO - Work done 2025-08-25 00:03:55,281 - __main__ - INFO - Got cancellation request for SGLang server