VirtualTam's bookmarks

1. Zarf Updates - Interactive Fiction Archive search feature

   2025-05-10
   game/interactive-fiction programming/python search-engine

   Permalink
   • IF Archive Search Page
   • searchlib/cli.py @240e47c
   • Sygil-Dev/whoosh-reloaded - A fast, featureful full-text indexing and searching library implemented in pure Python
2. Whoosh Reloaded - a fast, featureful full-text indexing and searching library implemented in pure Python

   2025-05-01
   programming/python query search-engine

   Permalink
   • Documentation
3. robots.txt - AI / LLM User-Agents: Blocking Guide

   2025-04-29
   network/http robots.txt web/crawling web/scraping

   Permalink
   • A summary of my bot defence systems
     • Butlerian Jihad - Blog posts on the topic of fighting off spam bots, search engine spiders and other non-humans wasting the precious resources we have on Earth
   • EmacsWiki's robots.txt
4. Archiving URLs | Gwern

   2025-04-09
   network/http personal-knowledge/bookmark personal-knowledge/digital-garden programming/go programming/html programming/java programming/python web web/scraping

   Permalink
   • Archiving Web Sites (LWN)
   • Robots.txt meant for search engines don’t work well for web archives

   Web page archive formats:

   • ARC (1996)
   • WARC (BnF, 2009)
     • WARC Specifications
     • The WARC Ecosystem

   Tools for crawling, scraping and archiving Web pages:

   • internetarchive/heritrix3 - Extensible, web-scale, archival-quality web crawler project (Java)
   • internetarchive/Zeno - State-of-the-art web crawler (Go)
     • internetarchive/gowarc - Read and write WARC files in Go
   • webrecorder/pywb - Web Archiving Toolkit for replay and recording of web archives (Python)

   Self-hosted solutions:

   • ArchiveBox - A self-hosted app that lets you preserve content from websites in a variety of formats
   • Wallabag - Save and classify articles, read them later
5. ICANN Update: Launching RDAP; Sunsetting WHOIS

   2025-02-27
   command-line ietf/rfc network/dns network/rdap programming/rust

   Permalink
   • icann/icann-rdap
   • The rdap Command
   • The Registration Data Access Protocol
     • Client Implementations
   • RFC 9082 - Registration Data Access Protocol (RDAP) Query Format
   • RFC 9536 - Registration Data Access Protocol (RDAP) Reverse Search
6. Zoekt - Fast trigram based code search

   2025-02-19
   programming/go search-engine source-code-management/git

   Permalink
   • FAQ
   • Zoekt Query Language Guide
   • isker/neogrok - A frontend for Zoekt
   • Query Language Guide by Neogrok
7. Limbo - A work-in-progress, in-process OLTP database management system, compatible with SQLite.

   2024-12-17
   asynchronous database/sqlite linux/kernel programming/rust

   Permalink
   • Serverless Runtime / Database Co-Design With Asynchronous I/O (PDF)
   • In search of a faster SQLite
8. EphemeraSearch - Your ancestors had mail

   2024-12-03
   database/postgresql genealogy history image/optical-character-recognition linux/container/kubernetes mail/postcard programming/go search-engine

   Permalink
   • About EphemeraSearch
   • EphemeraSearch on Github
   • Enix Live Show - Comment CNPG et Kubernetes ont sauvé ma startup !
   • PoZoK - PostgreSQL on ZFS on Kubernetes
9. A look at search engines with their own indexes

   2024-11-28
   alternative network/http search-engine search-engine/search-engine-optimization web

   Permalink
   • Very Alternative Search Engines
   • Neat search engines you may not have heard of
   • Lots of great new search engines popping up that search the rest of the web that Google tends to ignore.
   • How bad are search results? Let's compare Google, Bing, Marginalia, Kagi, Mwmbl, and ChatGPT
10. ast-grep - Command-line tool for structural code search & rewrite

    2024-10-30
    abstract-syntax-tree command-line parser programming/rust

    Permalink
    • ast-grep/ast-grep
11. How to use PostgreSQL for (military) geoanalytics tasks

    2024-03-31
    country/russia country/ukraine database/postgresql geography/gis map nasa/earthdata war

    Permalink
    • From Map to Reality: Using PostGIS in Warfare (PDF)
    • NASA Earthdata - Full and open access to NASA’s collection of Earth science data for understanding and protecting our home planet
    • Visible Infrared Imaging Radiometer Suite (VIIRS) Active Fire detection
    • PostgreSQL - GiST - Generalized Search Tree
    • PostgreSQL - SP-GiST - Space-partitioned GiST
    • H3 - Hexagonal hierarchical geospatial indexing system
    • zachasme/h3-pg - PostgreSQL bindings for H3
12. Regex engine internals as a library

    2023-07-14
    programming/c++ programming/go programming/regular-expressions programming/rust

    Permalink
    • rust-lang/regex
    • google/re2
    • BurntSushi/rebar - A biased barometer for gauging the relative speed of some regex engines on a curated set of tasks
    • Resources about implementing regular expression search efficiently, Russ Cox
13. Create an advanced search engine with PostgreSQL

    2023-07-14
    database/postgresql programming/sql search-engine text

    Permalink
    • Postgres Full Text Search vs the rest (MeiliSearch, OpenSearch/Elasticsearch, SQlite FTS, Typesense)
    • Optimizing Full Text Search with Postgres tsvector Columns and Triggers
    • Word separators for Postgres full text search with Rails
    • Using PostgreSQL Full Text Search With Golang
    • PostgreSQL: Full Text Search
    • PostgreSQL: Preferred Index Types for Text Search
14. Social Media Analytics & Tracking Query Parameters

    2023-06-11
    internet/analytics internet/browser network/url privacy search-engine/search-engine-optimization

    Permalink
    • UTM Parameters
    • Google Ads - Conversion Tracking
    • Websites use URLs to track you. Here’s how to stop them.
    • Firefox’s privacy crusade now targets a key form of tracking
    • ClearURLs - An addon to remove tracking elements from URLs to help protect your privacy
    • iOS 17 automatically removes tracking parameters from links you click on
15. Image Stacks and iPhone Racks - Building an Internet Scale Meme Search Engine | FindThatMeme.com Blog

    2023-02-14
    apple image image/optical-character-recognition meme phone

    Permalink
16. NucliaDB - a cloud-native database for unstructured data and vector, text, paragraphs and relations indexing

    2022-11-19
    database/nosql natural-language-processing network/grpc network/protocol-buffers programming/python programming/rust search-engine

    Permalink
    • https://github.com/nuclia/nucliadb
    • https://nuclia.com/building-nuclia/first-release/
    • https://2022.ploneconf.org/schedule/talks/nucliadb-oss-cloud-native-database-for-unstructured-data
    • https://gist.github.com/bloodbare/8381bfd0b11d94f2e8592b1101e695b5
    • https://github.com/nuclia/demos/
17. astpath: A command-line search utility for Python ASTs using XPath syntax.

    2022-08-02
    abstract-syntax-tree command-line parser programming/python programming/xml/xpath

    Permalink
18. PISA - Performant Indexes and Search for Academia

    2022-07-03
    programming/c++ programming/python search-engine

    Permalink
    • https://github.com/pisa-engine/pisa
    • http://ceur-ws.org/Vol-2409/docker08.pdf
    • https://github.com/ot/ds2i/
19. Falsehoods Programmers Believe About Search - OpenSource Connections

    2022-05-31
    programming search-engine web

    Permalink
20. Can I search for a Unicode combining character in Vim?

    2022-03-04
    editor/vim text/unicode

    Permalink