ReAct Agent Tools Documentation

Use this to analyze and understand multiple images using Claude's vision capabilities. If you have local image files, you can use generate_tmp_public_url tool to get public URLs first: 1. Call generate_tmp_public_url for each local image file 2. Collect all returned URLs into a list 3. Use the list of URLs as the img_urls parameter for this function Data Visualization Integration: When the user's input indicates a need for comparison or data visualization (e.g., "compare the values in these charts", "extract and plot the data from these images"), this function can return data in a format suitable for Plotly visualization. The returned data will be a dictionary with a special key "__plotly_data__" containing: { "__plotly_data__": { "data": [...], # Plotly data array "layout": {...} # Plotly layout object } } You can then pass this data to create_plotly_chart to generate an interactive chart. Supported image formats: - JPEG, PNG, GIF, WebP - Maximum file size: 5MB per image - Recommended size: No more than 1568 pixels in either dimension - Very small images (under 200 pixels) may degrade performance - Can analyze up to 20 images per request Capabilities: - Analyzing charts, graphs, and diagrams - Reading and understanding text in images - Describing visual content and scenes - Comparing multiple images in a single request - Answering questions about image details - Identifying relationships between images - Extracting data from charts for visualization Limitations: - Cannot identify or name specific people - May have reduced accuracy with low-quality or very small images - Limited spatial reasoning abilities - Cannot verify if images are AI-generated - Not designed for medical diagnosis or healthcare applications Args: img_urls: List of URLs to the image files (can be generated using generate_tmp_public_url for local files) user_input: Question or instruction about the image content(s) Returns: str: Claude's analysis of the image(s) based on the query, or Plotly-compatible data structure if visualization is needed

Use this to chat with a PDF file. User can ask about any text, pictures, charts, and tables in PDFs that is provided. Some sample use cases: - Analyzing financial reports and understanding charts/tables - Extracting key information from legal documents - Translation assistance for documents - Converting document information into structured formats Data Visualization Integration: When the user's input indicates a need for comparison or data visualization (e.g., "compare the quarterly profits", "show the trend of sales"), this function can return data in a format suitable for Plotly visualization. The returned data will be a dictionary with a special key "__plotly_data__" containing: { "__plotly_data__": { "data": [...], # Plotly data array "layout": {...} # Plotly layout object } } You can then pass this data to create_plotly_chart to generate an interactive chart. If you have a local PDF file, you can use generate_tmp_public_url tool to get a public URL first: 1. Call generate_tmp_public_url with your local PDF file path 2. Use the returned URL as the pdf_url parameter for this function Args: pdf_url: the URL to the PDF file (can be generated using generate_tmp_public_url for local files) user_input: the user's input Returns: str: Analysis result or Plotly-compatible data structure if visualization is needed

Create an interactive Mermaid diagram visualization and return its URL. This URL should be provided to the user, as they will need to access it to view the interactive diagram in their web browser. 使用 create_mermaid_diagram 的情境： - 提到「流程圖」、「架構圖」、「關係圖」等字眼 - 需要展示系統架構（flowchart） - 需要說明操作流程（flowchart） - 需要展示時序互動（sequence diagram） - 需要展示狀態轉換（state diagram） - 需要展示類別關係（class diagram） - 需要展示實體關係（ER diagram） - 需要展示專案時程（gantt chart） - 需要展示使用者旅程（journey） - 需要展示需求關係（requirement diagram） - 需要展示資源分配（pie chart） Supported Diagram Types: 1. Flowcharts (graph TD/LR): - System architectures - Process flows - Decision trees - Data flows 2. Sequence Diagrams (sequenceDiagram): - API interactions - System communications - User interactions - Message flows 3. Class Diagrams (classDiagram): - Software architecture - Object relationships - System components - Code structure 4. State Diagrams (stateDiagram-v2): - System states - Workflow states - Process states - State transitions 5. Entity Relationship Diagrams (erDiagram): - Database schemas - Data relationships - System entities - Data models 6. User Journey Diagrams (journey): - User experiences - Customer flows - Process steps - Task sequences 7. Gantt Charts (gantt): - Project timelines - Task schedules - Resource allocation - Milestone tracking 8. Pie Charts (pie): - Data distribution - Resource allocation - Market share - Component breakdown 9. Requirement Diagrams (requirementDiagram): - System requirements - Dependencies - Specifications - Constraints Example Mermaid syntax for a simple flowchart: ``` graph TD A[開始] --> B{是否有資料?} B -->|是| C[處理資料] B -->|否| D[取得資料] C --> E[結束] D --> B ``` Args: mermaid_data: String containing the Mermaid diagram definition title: Optional title for the diagram. If provided, must be in Traditional Chinese. For example: "系統流程圖" instead of "System Flowchart" Returns: str: URL for the interactive HTML visualization. This URL should be provided to the user, as they will need to access it to view the interactive diagram in their web browser.

Create an interactive Plotly visualization and return its URL. This URL should be provided to the user, as they will need to access it to view the interactive chart in their web browser. 使用 create_plotly_chart 的情境： - 提到「圖表」、「統計圖」、「視覺化」等字眼 - 需要呈現數據趨勢（折線圖） - 需要比較數值（長條圖、圓餅圖） - 需要展示分布情況（散點圖、熱力圖） - 需要顯示時間序列資料（時間軸圖） - 需要展示地理資訊（地圖） - 需要多維度資料分析（3D圖、氣泡圖） - 需要展示統計分布（箱型圖） - 需要展示累積趨勢（面積圖） - 需要互動式資料探索 Integration with Other Tools: This function can be used in conjunction with chat_with_imgs and chat_with_pdf when they return data suitable for visualization. When those tools detect a need for visualization, they will return a dictionary with a "__plotly_data__" key, which can be directly passed to this function. Example workflow: 1. User asks to analyze and visualize data from images/PDFs 2. chat_with_imgs or chat_with_pdf returns data with "__plotly_data__" key 3. Pass that data to this function to get an interactive visualization URL Supported Chart Types: - Line charts: For showing trends and time series data - Bar charts: For comparing values across categories - Pie charts: For showing proportions of a whole - Scatter plots: For showing relationships between variables - Heat maps: For showing patterns in matrix data - Box plots: For showing statistical distributions - Geographic maps: For showing spatial data - 3D plots: For showing three-dimensional data - Bubble charts: For showing three variables in 2D - Area charts: For showing cumulative totals over time The figure_data should be a dictionary containing plotly figure specifications with 'data' and 'layout'. Example: { 'data': [{ 'type': 'scatter', 'x': [1, 2, 3, 4], 'y': [10, 15, 13, 17] }], 'layout': { 'title': 'My Plot' } } Args: figure_data: Dictionary containing plotly figure specifications or output from chat_with_imgs/chat_with_pdf title: Optional title for the plot. If provided, must be in Traditional Chinese. For example: "台灣人口統計圖表" instead of "Taiwan Population Chart" Returns: str: URL for the interactive HTML visualization. This URL should be provided to the user, as they will need to access it to view the interactive chart in their web browser.

Use this to get the current time in local timezone.

Use this to get the days between two dates. Args: start_date: the start date, format: YYYY-MM-DD end_date: the end date, format: YYYY-MM-DD

Use this to generate high-quality images using DALL-E 3. Capabilities: - Creates photorealistic images and art - Handles complex scenes and compositions - Maintains consistent styles - Follows detailed prompts with high accuracy - Supports various artistic styles and mediums Best practices for prompts: - Be specific about style, mood, lighting, and composition - Include details about perspective and setting - Specify artistic medium if desired (e.g., "oil painting", "digital art") - Mention color schemes or specific colors - Describe the atmosphere or emotion you want to convey Limitations: - Cannot generate images of public figures or celebrities - Avoids harmful, violent, or adult content - May have inconsistencies with hands, faces, or text - Cannot generate exact copies of existing artworks or brands - Limited to single image generation per request Args: user_input: Detailed description of the image you want to generate. Be specific about style, content, and composition. Returns: str: URL to the generated image, or error message if generation fails

Generate a temporary public URL for a local file. The file will be automatically deleted after 7 days. Args: file_path: The path to the local file you want to make publicly accessible Returns: str: A public URL that can be used to access the file for 7 days Raises: FileNotFoundError: If the specified file does not exist

Use this to scrape the web. as it provides better results for video content. Args: url: the url to scrape

Use this to search the web when you need up-to-date information or when your knowledge is insufficient. This tool uses Perplexity to perform web searches and provides detailed answers with citations. Args: user_input: The search query or question you want to find information about. Returns: str: A detailed answer based on web search results, including citations to source materials