# Copilot The LLM Scraper enables retrieval of Copilot responses and returns a structured JSON output. *** #### **API Setup** **Endpoint** ``` https://llm-scraper.netnut.io/search ``` *** #### **Authentication** Use HTTP Basic Authentication: ``` Authorization: Basic ``` Where `` is the Base64-encoded string of your NetNut credentials. *** #### **Request Body Schema** ```json { "prompt": "Explain SEO ranking factors in 2025", // Required "engine": "copilot", // Required "mode": "chat", // Optional "country": "us", // Optional "throttling": true // Optional, default true } ``` **Field Descriptions** | Field | Description | Parameter Type | | -------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------- | | **prompt** | Main prompt sent to Copilot. Max length: 4096 chars. | Required | | **country** | Target geographic region for contextual responses. | Optional | | **engine** | The chatbot engine to use. Currently "chatgpt", "google\_ai\_mode", "perplexity", "gemini" and "copilot" are supported. | Required (default: chatgpt) | | **mode** | Copilot chat mode. Must be one of: `chat`, `reasoning`, `smart`, `study`. | Optional (default: chat) | | **throttling** | Controls automated throttling behavior. Enabled by default to help maintain a high success rate during heavy traffic or platform-side limitations. Set to false to opt out. | Optional (default: true) | *** #### **Example Usage** {% tabs %} {% tab title="cURL" %} ```bash curl -X POST https://llm-scraper.netnut.io/search \ -H "Content-Type: application/json" \ -U "username:password" \ -d '{ "prompt": "Give me SEO trends for 2026", "engine": "copilot", "mode": "chat", "country": "uk", "throttling": true }' ``` {% endtab %} {% tab title="Python" %} ```python import requests from base64 import b64encode headers = { "Authorization": "Basic " + b64encode(b"username:password").decode(), "Content-Type": "application/json" } body = { "prompt": "Give me SEO trends for 2026", "country": "us", "engine": "copilot", "mode": "chat", } response = requests.post("https://llm-scraper.netnut.io/search", headers=headers, json=body) print(response.json()) ``` {% endtab %} {% tab title="JavaScript" %} ```javascript const username = 'your_username'; const password = 'your_password'; const auth = Buffer.from(`${username}:${password}`).toString('base64'); const headers = { 'Authorization': `Basic ${auth}`, 'Content-Type': 'application/json' }; const body = { prompt: 'Give me SEO trends for 2026', engine: "copilot", mode: "chat", country: 'us' }; fetch('https://llm-scraper.netnut.io/search', { method: 'POST', headers: headers, body: JSON.stringify(body) }) .then(response => response.json()) .then(data => console.log(data)) .catch(error => console.error('Error:', error.message)); ``` {% endtab %} {% endtabs %} *** #### **Response Format** The Copilot Scraper always returns **structured JSON**.\ No HTML is returned. **Example JSON Response** ```json { "timestamp": "2026-04-20T12:19:21.231Z", "traceID": "a1b2c3d4-e5f6-7a8b-9c0d-e1f2a3b4c5d6", "request_duration": 14.28, "process_duration": 14.28, "scraper": "Copilot", "response": { "prompt": "whats the best residential proxy provider?", "text": "The leading SEO tips for 2026 center on AI-readiness, entity-based optimization, technical excellence, and delivering deeply useful, intent-matched content. Winning in 2026 means helping AI interpret your expertise: build topic clusters and entity relationships, use structured data and schema, and create passage-ready content so LLMs can extract accurate snippets. Strengthen foundational SEO by improving crawlability, internal linking, and E-E-A-T signals. Refresh existing content before creating new — updating older pages often produces faster ranking gains. Improve Core Web Vitals and optimize for mobile-first indexing.", "text_markdown": "**The leading SEO tips for 2026 center on AI-readiness, entity-based optimization, technical excellence, and delivering deeply useful, intent-matched content.**\n\n### 1. Optimize for AI-Driven Search & LLM Discovery\n- Build **topic clusters** and **entity relationships** instead of isolated keyword pages.\n- Use **structured data** and schema to make content easy for AI to cite.\n- Create **passage-ready content** so LLMs can extract accurate snippets.\n\n### 2. Strengthen Foundational SEO (Technical + E-E-A-T)\n- Improve crawlability, internal linking, and site architecture.\n- Demonstrate **E-E-A-T** to become a trusted source for both humans and AI.\n\n### 3. Refresh Existing Content Before Creating New\n- Prioritize pages already ranking on page 2 or low page 1 for quick wins.\n\n### 4. Enhance Technical Performance & User Experience\n- Improve **Core Web Vitals** (speed, interactivity, stability).\n- Optimize for mobile-first indexing.", "citations_found": true, "citations": [ { "id": 1, "url": "https://moz.com/blog/top-seo-tips-2026-whiteboard-friday", "title": "Top SEO Tips For 2026 - Moz", "section": "citations" }, { "id": 2, "url": "https://sitebulb.com/resources/guides/seo-in-2026-17-expert-tips-predictions/", "title": "SEO in 2026: 17 Expert Tips & Predictions - Sitebulb", "section": "citations" }, { "id": 3, "url": "https://www.bay20.com/10-practical-seo-tips-that-still-work-in-2026/", "title": "10 Practical SEO Tips That Still Work in 2026 - bay20.com", "section": "citations" }, { "id": 4, "url": "https://www.ranktracker.com/blog/improve-seo-2026-guide/", "title": "How to Improve SEO in 2026: A Complete Guide to Boost Rankings, Traffic ...", "section": "citations" } ], "inline_citations": [ { "id": 1, "url": "https://moz.com/blog/top-seo-tips-2026-whiteboard-friday", "title": "Top SEO Tips For 2026 - Moz", "text_anchor": "Build topic clusters and entity relationships instead of isolated keyword pages. Use structured data and schema to make content easy for AI to cite." }, { "id": 2, "url": "https://sitebulb.com/resources/guides/seo-in-2026-17-expert-tips-predictions/", "title": "SEO in 2026: 17 Expert Tips & Predictions - Sitebulb", "text_anchor": "Improve crawlability, internal linking, and site architecture. Demonstrate E-E-A-T to become a trusted source for both humans and AI." } ], "tables": [ { "title": "Quick Comparison: What Matters Most in 2026", "table_columns": ["Priority", "Why It Matters"], "table_rows": [ [{"value": "AI & Entity Optimization"}, {"value": "AI search relies on semantic understanding"}], [{"value": "Technical SEO"}, {"value": "Crawlability + speed directly impact rankings"}], [{"value": "E-E-A-T"}, {"value": "AI and users trust authoritative sources"}], [{"value": "Content Refreshing"}, {"value": "Faster ranking gains than new content"}], [{"value": "User Experience"}, {"value": "Mobile-first indexing + engagement signals"}] ] } ] } } ``` --- # Agent Instructions: Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter: ``` GET https://help.netnut.io/netnut-documentation/netnut-scraper-apis/llm-scraper/copilot.md?ask= ``` The question should be specific, self-contained, and written in natural language. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.