$ python add_site.py --yes https://www.budgetbytes.com
Working directory: /app
------------------------------------------------------------
============================================================
Adding support for: budgetbytes.com
Using model: deepseek/deepseek-v3.2
============================================================
============================================================
COMPREHENSIVE SITE ANALYSIS
Model: deepseek/deepseek-v3.2
============================================================
Step 0: Gathering site data...
Fetching URLs from homepage...
Found 100 URLs on homepage
Fetching sitemap...
Found sitemap index with 6 sitemaps
Found 2141 URLs in sitemap (115 category URLs)
Analyzing navigation structure...
Found 24 navigation items (14 likely categories)
Step 1: Analyzing URL patterns...
Recipe pattern: ^https://www\.budgetbytes\.com/[a-z0-9-]+(?:-[a-z0-9-]+)*/$
URL structure: flat
Confidence: high
Step 2: Analyzing category structure...
LLM found categories: True
LLM proposed 59 categories
Evidence provided: 59 items
Step 2.5: Validating discovered categories (HTTP check)...
Validating 59 categories...
✓ Air Fryer Recipes: 28 recipes
✓ Appetizers: 29 recipes
✓ Asian Recipes: 31 recipes
✓ Chicken Baked: 17 recipes
✓ Beans and Grains: 34 recipes
✓ Beans: 30 recipes
✓ Beef Meal Prep: 18 recipes
✓ Beef Recipes: 26 recipes
✓ Slow Cooker Beef: 18 recipes
✓ Beef Soup: 12 recipes
✓ Bowl Meals: 22 recipes
✓ Bread: 24 recipes
✓ Breakfast Baked Goods: 18 recipes
✓ Breakfast Meal Prep: 18 recipes
✓ Breakfast Recipes: 38 recipes
✓ Budget Friendly Meal Prep: 28 recipes
✓ Chicken Breasts: 43 recipes
✓ Chicken Meal Prep: 18 recipes
✓ Chicken Pasta: 22 recipes
✓ Chicken Recipes: 59 recipes
✓ Chicken Salad: 18 recipes
✓ Slow Cooker Chicken: 18 recipes
✓ Chicken Soup: 18 recipes
✓ Chicken Thighs: 17 recipes
✓ Chicken Wings: 12 recipes
✓ Christmas Recipes: 47 recipes
✓ Cold Breakfast Recipes: 18 recipes
✓ Comfort Food: 49 recipes
✓ Cost Per Recipe: 34 recipes
✓ Cost Per Serving: 23 recipes
✓ Dairy Free: 18 recipes
✓ Dessert: 46 recipes
✓ Dips: 33 recipes
✓ Dollar Store Dinners: 10 recipes
✓ Dressing: 21 recipes
✓ Drinks: 24 recipes
✓ East Asian Recipes: 17 recipes
✓ Easter Recipes: 31 recipes
✓ Eastern European Recipes: 9 recipes
✓ Egg Free: 22 recipes
✓ Eggs Breakfast: 19 recipes
✓ Extra Bytes: 18 recipes
✓ Fall Recipes: 43 recipes
✓ Fruit: 31 recipes
✓ Giveaways: 7 recipes
✓ Global Recipes: 18 recipes
✓ Gluten Free: 22 recipes
✓ Holiday Recipes: 35 recipes
✓ Hot Breakfast Recipes: 18 recipes
✓ How To: 25 recipes
✓ One Pot Meals: 36 recipes
✓ Pasta Recipes: 60 recipes
✓ Recipe Roundups: 24 recipes
✓ Recipes under $10: 38 recipes
✓ Recipes: 53 recipes
✓ Salad Recipes: 55 recipes
✓ Sandwich Recipes: 39 recipes
✓ Soup Recipes: 39 recipes
✓ Vegetarian Recipes: 51 recipes
Validated: 59 categories
Step 3: Validating configuration...
Pattern valid: False
Unwanted valid: False
Categories valid: False
Issues: ['Recipe pattern incorrectly matches non-recipe URLs like /kitchen-basics/ and /privacy-policy/', 'Unwanted patterns list includes patterns that should be excluded but are not comprehensive enough', 'Test URLs show /kitchen-basics/ and /privacy-policy/ should NOT match but current pattern would match them', "Category pages list includes many categories but test shows /contact/, /author/, /about/ should be filtered but aren't in unwanted patterns", "Recipe pattern doesn't account for variations like /how-to- URLs that should be excluded"]
============================================================
ANALYSIS SUMMARY
============================================================
Recipe pattern: ^https://www\.budgetbytes\.com/[a-z0-9-]+(?:-[a-z0-9-]+)*/$
Unwanted patterns: 15 patterns
Has categories: True
Categories found: 59
- Air Fryer Recipes
- Appetizers
- Asian Recipes
- Chicken Baked
- Beans and Grains
- Beans
- Beef Meal Prep
- Beef Recipes
- Slow Cooker Beef
- Beef Soup
... and 49 more
Confidence: high
Validation issues:
⚠️ Recipe pattern incorrectly matches non-recipe URLs like /kitchen-basics/ and /privacy-policy/
⚠️ Unwanted patterns list includes patterns that should be excluded but are not comprehensive enough
⚠️ Test URLs show /kitchen-basics/ and /privacy-policy/ should NOT match but current pattern would match them
⚠️ Category pages list includes many categories but test shows /contact/, /author/, /about/ should be filtered but aren't in unwanted patterns
⚠️ Recipe pattern doesn't account for variations like /how-to- URLs that should be excluded
Suggestions:
💡 Update recipe pattern to be more specific to actual recipe pages
💡 Add patterns to exclude common non-recipe sections like /contact/, /author/, /about/
💡 Ensure unwanted patterns cover all non-recipe sections identified in test URLs
💡 Review category pages for completeness against actual website structure
============================================================
GENERATED SCRAPER CODE
============================================================
"""Budgetbytes scraper.
Generated by add_site.py with comprehensive LLM analysis.
URL structure: flat
Recipe URLs follow a flat structure with a single slug (e.g., /cowboy-caviar/). Non-recipe URLs include category pages, policy pages, FAQ, how-to guides, list articles (starting with numbers), and query parameters.
"""
import re
from recipe_urls._abstract import AbstractScraper
class BudgetbytesScraper(AbstractScraper):
RECIPE_PATTERN = re.compile(r"^https://www\.budgetbytes\.com/[a-z0-9-]+(?:-[a-z0-9-]+)*/$")
UNWANTED_PATTERNS = [
re.compile(pattern)
for pattern in ['^https://www\\.budgetbytes\\.com/category/', '^https://www\\.budgetbytes\\.com/\\?', '^https://www\\.budgetbytes\\.com/kitchen-basics/$', '^https://www\\.budgetbytes\\.com/privacy-policy/$', '^https://www\\.budgetbytes\\.com/faq/$', '^https://www\\.budgetbytes\\.com/terms-conditions/$', '^https://www\\.budgetbytes\\.com/welcome-to-budget-bytes/$', '^https://www\\.budgetbytes\\.com/random/$', '^https://www\\.budgetbytes\\.com/how-to-', '^https://www\\.budgetbytes\\.com/\\d+-', '^https://www\\.budgetbytes\\.com/leftover-', '^https://www\\.budgetbytes\\.com/\\d+-foods-', '^https://www\\.budgetbytes\\.com/contact/$', '^https://www\\.budgetbytes\\.com/author/', '^https://www\\.budgetbytes\\.com/about/$']
]
# Category pages for bulk import with category selection
CATEGORY_PAGES = {
'Air Fryer Recipes': '/category/recipes/air-fryer/',
'Appetizers': '/category/recipes/appetizers/',
'Asian Recipes': '/category/recipes/global/asian/',
'Beans': '/category/recipes/beansandgrains/beans/',
'Beans and Grains': '/category/recipes/beansandgrains/',
'Beef Meal Prep': '/category/extra-bytes/budget-friendly-meal-prep/beef-meal-prep/',
'Beef Recipes': '/category/recipes/meat/beef/',
'Beef Soup': '/category/recipes/soup/beef-soup/',
'Bowl Meals': '/category/recipes/bowl-meals/',
'Bread': '/category/recipes/bread/',
'Breakfast Baked Goods': '/category/recipes/breakfast/breakfast-baked-goods/',
'Breakfast Meal Prep': '/category/extra-bytes/budget-friendly-meal-prep/breakfast-meal-prep/',
'Breakfast Recipes': '/category/recipes/breakfast/',
'Budget Friendly Meal Prep': '/category/extra-bytes/budget-friendly-meal-prep/',
'Chicken Baked': '/category/recipes/meat/chicken/chicken-baked/',
'Chicken Breasts': '/category/recipes/meat/chicken/chicken-breasts/',
'Chicken Meal Prep': '/category/extra-bytes/budget-friendly-meal-prep/chicken-meal-prep/',
'Chicken Pasta': '/category/recipes/pasta/chicken-pasta/',
'Chicken Recipes': '/category/recipes/meat/chicken/',
'Chicken Salad': '/category/recipes/meat/chicken/chicken-salad/',
'Chicken Soup': '/category/recipes/soup/chicken-soup/',
'Chicken Thighs': '/category/recipes/meat/chicken/chicken-thighs/',
'Chicken Wings': '/category/recipes/meat/chicken/chicken-wings/',
'Christmas Recipes': '/category/recipes/holiday-recipes/christmas-recipes/',
'Cold Breakfast Recipes': '/category/recipes/breakfast/cold-breakfast-recipes/',
'Comfort Food': '/category/recipes/comfort-food/',
'Cost Per Recipe': '/category/recipes/cost-per-recipe/',
'Cost Per Serving': '/category/recipes/cost-per-serving/',
'Dairy Free': '/category/recipes/dairy-free/',
'Dessert': '/category/recipes/dessert/',
'Dips': '/category/recipes/dips/',
'Dollar Store Dinners': '/category/recipes/dollar-store-dinners/',
'Dressing': '/category/recipes/dressing/',
'Drinks': '/category/recipes/drinks/',
'East Asian Recipes': '/category/recipes/global/asian/east-asian/',
'Easter Recipes': '/category/recipes/holiday-recipes/easter-recipes/',
'Eastern European Recipes': '/category/recipes/global/eastern-european/',
'Egg Free': '/category/recipes/egg-free/',
'Eggs Breakfast': '/category/recipes/breakfast/eggs-breakfast/',
'Extra Bytes': '/category/extra-bytes/',
'Fall Recipes': '/category/recipes/seasonal/fall-recipes/',
'Fruit': '/category/recipes/fruit/',
'Giveaways': '/category/giveaways/',
'Global Recipes': '/category/recipes/global/',
'Gluten Free': '/category/recipes/gluten-free/',
'Holiday Recipes': '/category/recipes/holiday-recipes/',
'Hot Breakfast Recipes': '/category/recipes/breakfast/hot-breakfast-recipes/',
'How To': '/category/how-to/',
'One Pot Meals': '/category/recipes/one-pot/',
'Pasta Recipes': '/category/recipes/pasta/',
'Recipe Roundups': '/category/extra-bytes/recipe-roundups/',
'Recipes': '/category/recipes/',
'Recipes under $10': '/category/recipes/cost-per-recipe/recipes-under-10/',
'Salad Recipes': '/category/recipes/salads/',
'Sandwich Recipes': '/category/recipes/sandwich/',
'Slow Cooker Beef': '/category/recipes/slow-cooker/slow-cooker-beef/',
'Slow Cooker Chicken': '/category/recipes/slow-cooker/slow-cooker-chicken/',
'Soup Recipes': '/category/recipes/soup/',
'Vegetarian Recipes': '/category/recipes/vegetarian/',
}
@classmethod
def host(cls):
return "budgetbytes.com"
✅ Saved: /app/data/scrapers/budgetbytes.py
============================================================
SUCCESS
============================================================
Site added: budgetbytes.com
Test it:
python import_site.py https://www.budgetbytes.com --dry-run
This site has 59 categories configured.
Users can select categories when bulk importing.
------------------------------------------------------------
✅ Completed successfully