VirtualTam's bookmarks

1. robots.txt - AI / LLM User-Agents: Blocking Guide 2025-04-29

   tags: network/http, robots.txt, web/crawling, web/scraping

   [ permalink ]

   • A summary of my bot defence systems
     • Butlerian Jihad - Blog posts on the topic of fighting off spam bots, search engine spiders and other non-humans wasting the precious resources we have on Earth
   • EmacsWiki's robots.txt
2. A different approach to blocking bad webbots by IP address 2025-04-28

   tags: feed, network/autonomous-system, network/http, network/ip, web/crawling, web/scraping

   [ permalink ]

   • Now a bit about feed readers
   • A deeper dive into mapping web requests via ASN, not by IP address
   • Notes on blocking spam by filtering on ASN
3. This blog is hosted on a Nintendo Wii 2025-04-28

   tags: bsd/netbsd, network/http, nintendo/wii

   [ permalink ]
4. Archiving URLs | Gwern 2025-04-09

   tags: network/http, personal-knowledge/bookmark, personal-knowledge/digital-garden, programming/html, web, web/scraping

   [ permalink ]
5. Principles Wiki - A place to collect, examine, and discuss software design knowledge in a systematic way. 2025-03-16

   tags: programming/design-pattern, programming/object-oriented-programming, software-engineering

   [ permalink ]

   • Full List of Principles
   • Easy To Use And Hard To Misuse
6. HAProxy as Egress Controller 2025-02-13

   tags: network/http, network/reverse-proxy, network/reverse-proxy/haproxy

   [ permalink ]
7. Cloudflare is blocking NewsBlur RSS feed fetchers 2025-02-08

   tags: cloud-computing/cloudflare, curl, feed/rss, network/http

   [ permalink ]

   • 403 using curl
   • Cloudflare Radar - Verified Bots
8. Git - Wire Protocol V2 2025-01-20

   tags: network/http, network/ssh, source-code-management/git

   [ permalink ]

   • Gitlab - Configuring Git Protocol v2
   • Github - Git Wire Protocol v2 Support, 2018-11-08
   • Introducing Git protocol version 2, 2018-05-18
   • Highlights from Git 2.26, 2020-03-22
9. CoreDNS - header plugin 2025-01-12

   tags: network/dns, network/http

   [ permalink ]

   • #3690 - CoreDNS as dns server, when doing "nslookup domain" command, I Got ";; Got recursion not available from xx.xx.xx.xx, trying next server"
   • #3835 - [investigation] Got recursion not available from ... trying next server
10. macos - DNS Resolution Fails for ping and curl, but not dig 2025-01-11

    tags: apple/macos, network/dns, network/http

    [ permalink ]

    • OSX Domain-Specific DNS Servers (/etc/resolver)
    • [macOS] need to support domain specific dns server (custom DNS)
    • Using scutil to set DNS for specific domains
    • How to query macOS DNS resolver from Terminal?
11. oha - HTTP load generator 2025-01-08

    tags: command-line, network/http, network/transport-layer-security, programming/rust

    [ permalink ]
12. Rust needs a web framework for lazy developers 2024-12-30

    tags: database, framework, network/http, programming/html, programming/rust, web

    [ permalink ]

    The benefits are there: you get wonderful type system, wonderful performance, and build times that give you back those coffee breaks you used to get while your code compiled.

    • ~ntietz/newt - Nicole's Web Toolkit
    • Are we web yet?
13. Model Context Protocol (MCP) - An open protocol that enables seamless integration between LLM applications and external data sources and tools 2024-11-30

    tags: editor, network/http, network/json-rpc, neural-network/large-language-model, programming/json, text

    [ permalink ]

    • Anthropic - Introducing the Model Context Protocol
    • modelcontextprotocol repositories
14. drf-spectacular - Sane and flexible OpenAPI 3 schema generation for Django REST framework 2024-11-28

    tags: framework/django, network/http/openapi, programming/api/rest, programming/python, type-hint

    [ permalink ]
15. A look at search engines with their own indexes 2024-11-28

    tags: alternative, network/http, search-engine, search-engine/search-engine-optimization, web

    [ permalink ]

    • Very Alternative Search Engines
    • Neat search engines you may not have heard of
    • Lots of great new search engines popping up that search the rest of the web that Google tends to ignore.
    • How bad are search results? Let's compare Google, Bing, Marginalia, Kagi, Mwmbl, and ChatGPT
16. ElectroPortis - The famous SGI/IRIX ElectroPaint(tm) screensaver by David Tristram, ported to Windows 2024-11-26

    tags: art, graphics/2d, programming/c, programming/c++, programming/javascript, screensaver

    [ permalink ]

    • Electropaint History (archived 2011-07-26)
    • Silicon Graphics - Irix - ElectroPaint screen saver
    • iamralpht/elektropaintjs - Electropaint screensaver from IRIX in JavaScript

          __    __
         /..\ /| |'-.
        .\_O/ || |   |   OH WOW
     _ /  `._ \|_|_.-'  FREE SOFTWARE
    | /  \__.`=._) (_
    |/ ._/  |"""""""""|  LET ME IN
    |'.  `\ |         |   ON THIS
    ;"""/ / |         |
     ) /_/| |.-------.|
    '  `-`' "         "
    

17. jwzhacks 2024-11-26

    tags: feed/rss, history, jwz, linux, network/http, programming/c, programming/perl, programming/php, unix

    [ permalink ]

    I have written a lot of software. I used to do it for fun. Then I started getting paid for it. Then it stopped being fun. Then I stopped getting paid for it. And now I write software sometimes for fun, but mostly in self-defense.

18. DNS over HTTPS 2024-11-25

    tags: cryptography/encryption, ietf/rfc, internet/browser/chrome, internet/browser/firefox, network/dns, network/http, network/transport-layer-security

    [ permalink ]

    RFCs:

    • RFC 7858 - Specification for DNS over Transport Layer Security (TLS)
    • RFC 8484 - DNS Queries over HTTPS (DoH)

    Resources:

    • Carsten Strotmann - doh.defaultroutes.de DNS-over-TLS/HTTPS Server
    • DNS over HTTPS - Cloudflare 1.1.1.1 Documentation

    Setting up a DNS-over-HTTPS server:

    • DNS over HTTPS - ArchWiki
    • DNS-over-TLS almost working - Træfik Forum
    • Using Google’s dns.google.com with CoreDNS

    Browser configuration:

    • Configure DoH on your browser - Cloudflare Documentation
    • Using custom DNS-over-HTTPS with CoreDNS and Google Chrome
    • Firefox DNS-over-HTTPS - Firefox Help
19. go-safeweb - Secure-by-default HTTP servers in Go 2024-11-23

    tags: network/http, programming/go, security, security/authentication

    [ permalink ]
20. WebCola - Javascript constraint-based graph layout 2024-11-22

    tags: graph, programming/c++, programming/javascript, programming/typescript, visualization

    [ permalink ]

    • cola.js - Constraint-Based Layout in the Browser
    • Adaptagrams - A library of tools and reusable code for adaptive diagramming applications
    • mjwybrow/adaptagrams