Skip to main content

SEO and Sitemap

Bellamy Book exposes SEO endpoints for search engines: a sitemap (XML) and robots.txt. Both are public (no auth) and help index the site and blogs.

Overview​

  • Sitemap index: https://bellamybook.com/sitemap.xml — Sitemap index that references:
    • sitemap-static.xml — Static pages (e.g. /landing, about, contact, auth, help).
    • sitemap-blogs.xml — Dynamic blog sitemap (proxied from backend): homepage, /blogs, and each published blog at /blogs/{slug}. Uses AppSettings:BaseUrl for absolute URLs. This URL is used instead of /api/sitemap/sitemap.xml in the index so crawlers (e.g. Google) avoid "General HTTP error" on the /api path; both URLs serve the same content.
  • Robots.txt: Served at /robots.txt (proxied from backend). Allows /, /blogs/, and /api/sitemap/ (so crawlers can fetch the blog sitemap). Disallows /admin/, /api/ (except sitemap), /settings/. References Sitemap: https://bellamybook.com/sitemap.xml.

API Endpoints (api/Sitemap)​

MethodEndpointDescription
GET/api/Sitemap/sitemap.xmlSitemap XML (application/xml). Homepage, blogs list, published blog URLs.
GET/api/Sitemap/robots.txtRobots.txt (text/plain). Allow/Disallow rules and Sitemap URL.

Both endpoints are AllowAnonymous. Base URL for links is taken from AppSettings:BaseUrl (or request host if not set).

Improving indexing (Google & Bing)​

If many pages show as “Discovered but not crawled” (Bing) or “Not indexed” (Google), do the following:

  1. Submit sitemaps

    • Google: Search Console → your property → Sitemaps → add https://bellamybook.com/sitemap.xml. For docs, add https://docs.bellamybook.com/sitemap.xml if you have a docs property.
    • Bing: Bing Webmaster Tools → Sitemaps → submit the same URLs.

  2. Request indexing for important URLs

    • In Google Search Console: URL Inspection → enter URL → Request indexing.
    • In Bing: URL Inspection → Request indexing (if the URL is “Discovered but not crawled”).

  3. Follow crawler guidelines

  4. Check the “3 reasons” in Google

    • In Search Console → Indexing → Pages → open Why pages aren’t indexed to see the exact reasons (e.g. “Crawled – currently not indexed”, “Duplicate”, “Soft 404”). Fix those (e.g. canonicals, redirects, or consolidating thin content).

  5. Docs site (docs.bellamybook.com)

    • Ensure the docs app is built with the sitemap plugin (see docusaurus.config.js). Submit https://docs.bellamybook.com/sitemap.xml in both Google and Bing if docs are a separate property.

  6. Frontend (SPA)

    • The main site is a client-rendered SPA. Google and Bing can run JavaScript; ensure important pages have unique <title>, <meta name="description">, and canonical URLs (e.g. via LandingPageSEO, BlogDetail meta). Avoid noindex on pages you want indexed.

Social sharing (Facebook, LinkedIn, Twitter, etc.)​

All pages (including the marketing landing at /landing) use the same Open Graph and Twitter Card image so that sharing https://bellamybook.com/landing shows the same preview image as https://bellamybook.com. The image is set in index.html with absolute URLs:

  • og:image and og:image:secure_url: https://bellamybook.com/og-image.png
  • twitter:image: same URL
  • Image size: 1200×630 px (recommended for Facebook/LinkedIn)

If the marketing URL (or another path) does not show the image when shared, social platforms may have cached an old version:

  1. Facebook: Sharing Debugger → enter https://bellamybook.com/landing → Scrape Again.
  2. LinkedIn: Post Inspector → enter the URL → Inspect.
  3. Twitter / X: Publish the link again; the card cache can take a short time to refresh.

After scraping again, new shares of that URL should show the same image as the homepage.

SEO checklist (what the project does)​

ItemStatus
Sitemap index at /sitemap.xml✅ Points to static + blog sitemaps
robots.txt✅ Allow /, /blogs/, /api/sitemap/; disallow admin/api/settings; Sitemap URL
Canonical URLs✅ Per-page via LandingPageSEO (/landing, about, contact, pricing, team, docs, download, blogs) and BlogDetail
Unique title + description✅ Public pages use LandingPageSEO; blog posts use BlogDetail meta
Open Graph & Twitter Card✅ In index.html (absolute URLs); per-page OG in LandingPageSEO and BlogDetail
og:image (1200×630)✅ og-image.png; og:image:secure_url in index.html
404 = noindex✅ NotFound page sets robots noindex,nofollow and restores on leave
Structured data✅ Organization JSON-LD in index.html; SoftwareApplication on the marketing landing; Article on blog posts
Document title✅ PageMetaData and LandingPageSEO set document.title in useEffect
Sitemap lastmod✅ Static sitemap and index use lastmod; blog sitemap from API has per-URL lastmod

When adding a new public page that should be indexed: add it to sitemap-static.xml, give it a route, and use LandingPageSEO (or equivalent) with title, description, and canonicalPath. Do not use noindex on pages you want in search results.