hlte.net

an open system for capturing, storing, and retrieving web content

Content Capture Methods

Manual highlighting via browser extension
Automatic capture of Bluesky liked posts
Email submission of URLs

Setup & Usage

Clone the daemon
- Or use the easy button & grab a pre-built package if available
Create a keyfile, by default named
```
.keyfile
```
in the daemon directory, with
```
tools/keygen
```
Adjust the daemon's listen port in the right environment's config file here (if necessary)
Run the daemon persistently
- If built yourself, via
```
mix run
```
- If via a pre-built package, via
```
hlte/bin/hlte start
```
Install the extension:
- click for Firefox (it is signed by Mozilla but self-distributed, so expect permission dialogs to appear); may also be directly loaded via
```
about:debugging
```
- Chrome must still be loaded in developer mode via these instructions (note the "Load Unpacked" screenshot)
  - The linked
```
.xpi
```
    for Firefox is just a
```
.zip
```
    file with a different extension, so you may extract & use that for Chrome as well
Open the extension's options and add your backend with the hexadecimal version of the key shown by
```
tools/keygen
```
You're setup! Go to any site and:
- Select text and click the icon that appears in the top-left of the page
- Right-click an image or video to annotate it directly
- Click the browser bar to annotate an entire page
Right-click the browser bar icon and select "Search..." to search your highlights

Additional Features

Email Integration

Setup SES to receive email with SNS hitting the /sns endpoint and backed by S3.

Raw message delivery is required on the SNS-to-your-HTTPS-endpoint call
You'll need to specify the whitelist of email addresses that are allowed to use this functionality
The subject of the email must be the primary URI to highlight & the body will be used as the highlight data

Bluesky Integration

Configure with your Bluesky account to automatically capture liked posts:

Set environment variables for Bluesky username and password
Configure Redis connection URL and HLTE authentication
Like posts on Bluesky to automatically capture them to your HLTE system

Persistence and Rendering

Content is automatically stored and rendered:

JSON metadata files stored for long-term archival
PDFs and screenshots created via Cloudflare Worker
Content stored in Cloudflare R2 storage

API Direct Access

Programmatically interact with your content:

Submit content: POST to /save endpoint with content data
Retrieve content: GET from /entries with appropriate query parameters
All requests require authentication via key

System Architecture

+---------------------+ Content +------------------+ Email Content | | Highlighting | | -------------> | Browser Extension | -------------> | | | | | | +-------------------------+ +---------------------+ | | | | | Elixir Daemon | <------+ Email (SNS/SES/S3) | | | | | | | +-------------------------+ | | +---------------------+ | | | | Liked Posts | | | Bluesky Account | -------------> | | | | +--------+---------+ +---------------------+ | | Content Events | (Redis Stream) v +-------------------+ | | | Persistence | | Service | | | +--------+----------+ | | URL Rendering | Requests v +-------------------+ | | +-----------------+ | Cloudflare | | | | Worker +----->+ R2 Storage | | | | (PDFs/Images) | +-------------------+ | | +-----------------+