agent-claw: automated task changes

This commit is contained in:
daniel
2026-05-06 18:55:16 -05:00
parent 38905bb1e9
commit 732b00fb66
8494 changed files with 2018127 additions and 4 deletions

View File

@@ -0,0 +1,8 @@
Public suffix list obtained from https://publicsuffix.org/.
We build a lookup map that for 90% of the cases can return the probable intended apex domain.
We're ignoring Punycode on purpose.
Whenever you pull a new version of the .dat file, don't forget to run build-map.py.
Not integrated as part of the build because this file will change only very rarely.

View File

@@ -0,0 +1,35 @@
#!/usr/bin/env python2.7
"""Script to build a lookup map from the lines in the public suffix data list.
See README.txt in this directory for more info.
"""
import re
import json
trie = {}
with open('public_suffix_list.dat', 'r') as f:
for line in f:
line = line.strip()
# All reasons to skip this line
if not line: continue
if line.startswith('//'): continue
if re.search('[^a-z0-9.]', line): continue
# *. at the start is the same as it not being there
if line.startswith('*.'): line = line[2:]
# Add to the trie
parts = line.split('.')
parts.reverse()
curr = trie
for part in parts:
curr = curr.setdefault(part, {})
with open('../lib/public-suffixes.ts', 'w') as o:
o.write('// This file has been generated using ../suffixes/build-map.py\n')
o.write('/* eslint-disable no-trailing-spaces, quote-props */\n')
o.write('export const publicSuffixes = %s;' % json.dumps(trie, indent=2))

File diff suppressed because it is too large Load Diff