> ## Documentation Index > Fetch the complete documentation index at: https://docs.scrapai.dev/llms.txt > Use this file to discover all available pages before exploring further. # Custom CSS Extractors > Site-specific extraction using CSS selectors Custom CSS extractors allow you to define precise selectors for extracting content from sites with unique structures. Use when generic extractors (newspaper, trafilatura) fail. ## When to Use Generic extractors fail or extract wrong content Site has unique/non-semantic HTML structure Need to extract custom fields (price, rating, category) E-commerce, job boards, real estate, forums Requires manual selector discovery and testing Site changes may break selectors ## Configuration ### For Article Content Use `CUSTOM_SELECTORS` in settings: ```json theme={null} { "settings": { "EXTRACTOR_ORDER": ["custom", "newspaper", "trafilatura"], "CUSTOM_SELECTORS": { "title": "h1.article-title", "content": "div.article-body", "author": "span.author-name", "date": "time.published-date", "category": "a.category-link", "tags": "div.tags a" } } } ``` ### For Structured Data Use callbacks for non-article content: ```json theme={null} { "callbacks": { "parse_product": { "extract": { "name": {"css": "h1.product-name::text"}, "price": {"css": "span.price::text"}, "rating": {"css": "span.rating::attr(data-rating)"} } } } } ``` See [Callbacks](/api/callbacks) for full schema. ## CSS Selector Syntax ### Basic Selectors ```json theme={null} // Element "css": "h1" // Class "css": "h1.article-title" // ID "css": "#main-title" // Multiple classes "css": "div.article.featured" // Descendant "css": "article div.content" // Direct child "css": "article > div.content" ``` ### Extract Text ```json theme={null} // Text content "css": "h1::text" // All text (including nested elements) "css": "div.content::text" ``` ### Extract Attributes ```json theme={null} // Image src "css": "img.main-image::attr(src)" // Link href "css": "a.external::attr(href)" // Data attribute "css": "span.rating::attr(data-rating)" // Class name "css": "div::attr(class)" ``` ### Multiple Matches ```json theme={null} { "css": "li.feature::text", "get_all": true } ``` Returns list: `["WiFi", "Bluetooth", "GPS"]` ## Standard vs Custom Fields ### Standard Fields (Database Columns) Map directly to `scraped_items` table: * `title` → scraped\_items.title * `content` → scraped\_items.content * `author` → scraped\_items.author * `date` → scraped\_items.published\_date ### Custom Fields (Metadata JSON) Any other field names stored in `metadata_json`: * `price`, `rating`, `category`, `brand`, etc. * Displayed in `show` command * Flattened in exports ## Selector Discovery Workflow ### Step 1: Inspect Page ```bash theme={null} scrapai inspect https://example.com/article --project proj ``` Saves HTML to `data/proj/spider/analysis/page.html` ### Step 2: Analyze Structure ```bash theme={null} scrapai analyze data/proj/spider/analysis/page.html ``` Shows: * h1/h2 elements with classes * Content containers by size * Date elements * Author elements ### Step 3: Test Selectors ```bash theme={null} scrapai analyze page.html --test "h1.article-title" scrapai analyze page.html --test "div.article-body" ``` Verifies selector matches correct element. ### Step 4: Search for Fields ```bash theme={null} scrapai analyze page.html --find "price" scrapai analyze page.html --find "rating" ``` Finds elements containing text. ### Step 5: Test on Multiple Pages ```bash theme={null} # Import spider with selectors scrapai spiders import spider.json --project proj # Test on 5 pages scrapai crawl spider --limit 5 --project proj # Verify results scrapai show spider 1 --project proj scrapai show spider 2 --project proj ``` ## Selector Best Practices Target main content element (not navigation/sidebar/footer) Selector should match ONE element per page Prefer specific classes (`.article-title`) over generic (`.title`) Test on multiple pages to verify consistency Prefer semantic tags (`
`, `