Startsida Utforska Hjälp
Logga in
darkrenaissance
/
webshop
2
0
Fork 0
Filer Ärenden 0 Pull-förfrågningar 0 Wiki
Träd: e82ee349b5
Grenar Taggar
webshop
/
README.md

README.md 10.0 KB
Historik Rå

hardware.dark.fi webshop

website for the Dark.Fi's hardware webshop.

python-based website integrating a webshop (Stripe, NOWPayments) and git-based workflow to update website content:

  • python 3.12
  • FastAPI
  • custom parser for md + yaml documents

setup

as the content repository is tracked with git, we're using git-annex to handle binary files. to setup this integration:

  • install git annex (see https://git-annex.branchable.com/install/)
    • if on Debian based systems, check the NeuroDebian channel for a more up-to-date version of git-annex

  • add a .gitattributes file with the following:

    *          annex.largefiles=nothing
*.png      filter=annex annex.largefiles=anything
*.jpg      filter=annex annex.largefiles=anything
*.pdf      filter=annex annex.largefiles=anything
*.epub     filter=annex annex.largefiles=anything

add more file type as needed. after this, we don't need to track binary files with git annex add but we can simply use git add.

  • initialize the repo: git init, git annex init
  • then work with files with git (git add, git mv, git rm, etc)

git annex useful commands

  • general sync: git annex sync --content; this syncs both from your computer to the remote server and vice-versa (eg it does a pull and then a push)
  • list files: git annex find
  • get copy of file: git annex get <filepath> (or without filepath to get all listed files)
  • remove file: git annex drop --force <filepath> (or without <filepath> to remove all listed files)
    • we need to use --force because by default git-annex tries to keep at least 1 more copy of the file in another repo, before deleting the file; in eg, it acts as an archive-focused system. since we might not care about that, passing --force bypasses the normal mechanism
    • NB: using git rm <filepath> will not remove the file from the repo, only the symlink git-annex created for it; we can check this by doing ls -l ./git/annex/objects/ and count how many items are in there

development

settings.toml

rename settings.sample.toml to settings.toml and fill out fields:

  • git_repo: set the path to the content folder (it can also be a relative path)
  • document_match: list of file extension to match
  • document_exclude: list of files to exclude
  • block_types: set necessary block types (for the content document); for starting, the default ones should be enough; the website's parser read from this list of block types to validate if a given block is "allowed"
  • local_db.filepath: path to local DB, eg file that contains the orders
  • order_upload.filepath: path to folder where new XLSX file order are exported to (these files are used to upload a shipping order on the Swiss Post website)
  • currency.default: default currency for Stripe
  • currency.now_payments: default currency for NOWPayments

.env

copy the file env.sample and rename it .env. then update its content accordingly:

  • ENV: set if environment is development or production (it affects some of the code logic)
  • CACHE: set enable or disable to cache CSS, JS and font files

  • CSRF_SECRET_KEY: generate a secret (a random string of characters) for the CSRF protection, see https://stackoverflow.com/a/54433731; for example:

    python
 
>>> import secrets
>>> secrets.token_hex(16)

  • SESSION_SECRET_KEY: see item above for how to generate a random string

  • STRIPE_SECRET_KEY: Stripe API key for production

  • STRIPE_TEST_SECRET_KEY: Stripe API key for dev environment. Put here your TEST API key. Beware that data in test environment differs from production

  • STRIPE_ENDPOINT_SECRET: Stripe webhook secret

    • important: use keys generated from Test mode while working on the website

  • NOWPAYMENTS_API_KEY: NOWPayments API key

  • NOWPAYMENTS_SANDBOX_IPN: NOWPayments IPN

  • NOWPAYMENTS_SANDBOX_API_KEY: NOWPayments Sandbox API key

  • NOWPAYMENTS_SANDBOX_IPN: NOWPayments Sandbox IPN

  • MAIL_USERNAME: email address

  • MAIL_PASSWORD: email password

  • MAIL_FROM: email sender address

  • MAIL_FROM_NAME: email sender name

  • MAIL_PORT: email port

  • MAIL_SERVER: email server

  • MAIL_STARTTLS: True|False

  • MAIL_SSL_TLS: True|False

  • MAIL_USE_CREDENTIALS: True|False

  • MAIL_VALIDATE_CERTS: True|False

  • CPU_CORE: set computer's number of CPU cores, on Linux run nproc --all; this is used by the build_image_cache.py command to generate images of various sizes for the frontend.

setup Python

  • python 3.12
  • make virtual environment: python -m venv env
  • activate virtual environment: source env/bin/activate
  • install packages with pip install -r requirements.txt

packages

whenever installing a new package, save it to requirements.txt by doing:

pip freeze > requirements.txt

server

to run the local web-server, type:

./dev-server.sh

to run the production server, do:

./server.sh

create image cache

we display images in various sizes so that the browser can pick the best image in terms of quality / size ratio — for instance by not downloading a high-res image prepared for a big screen if the browser request comes from a small screen device (eg. a phone). to deal with this, there a CLI script called build_image_cache.py at the root of the repo. it offers two commands:

before generating images make sure that the cache folder exists in static/images, otherwise images won't get written

  • build-all-images: this build all images across the content repo. if an image exists already in the ./static/images/cache/ folder, it will skip it. you can either remove it from the cache folder or rename the image to a new name to force-recreate it. run this command like so: python build_image_cache.py build-all-images '<path/to/content-repo>
  • build-images: this command is used in the git post-receive hook in the git bare repo (on the VPS). it checks if the last pushed commit contains also image files, and if so it creates the necessary sizes of each image for the website. check the post-receive files for the details

example:

python build_image_cache.py build-all-images <path/to/content-folder>

test stripe webhook (locally)

unlike with NOWPayments, we can locally test that the Stripe webhook works as intended — for instance that it triggers the order status update, the product inventory update, and the email confirmation.

to do so:

  • first install the Stripe CLI

  • then open a new shell and run:

    stripe listen --load-from-webhooks-api --forward-to http://127.0.0.1:5014

where http://127.0.0.1:5014 is your local server port (which is the one set in dev-server.sh). this command listen to our "online" Stripe webhook for any event, and redirects them all to our local environment.

when you run Stripe CLI locally, it might print the webhook signature valid for our local environment — make sure to add it or replace it to the field STRIPE_TEST_ENDPOINT_SECRET in the .env file.

some refs:

aftewards, run whatever you need to test.

server setup

installation

  • make new unix user from root (sudo adduser)
  • add ssh pubkey to new user's home under ~/.ssh/authorized_keys, set permissions (https://superuser.com/a/925859)
  • add newly created user under sudo group (sudo adduser <user> sudo)

  • update /etc/ssh/ssh_config by changing or adding values to the following:

    # disable ssh login for root user
PermitRootLogin no
ChallengeResponseAuthentication no
PasswordAuthentication no
UsePAM no

# remove password-based login for any user
AuthenticationMethods publickey
PubkeyAuthentication yes

# limit ssh access to specified users (IMPORTANT: this will block any other previously authorized user to login via ssh)
AllowUsers <user-1> <user-2>

# disable empty password
PermitEmptyPasswords no

# disable X11Forwarding
X11Forwarding no

then restart sshd with sudo systemctl restart sshd

  • install ufw and enable SSH + Nginx
  • install git
  • install pyenv (https://bgasparotto.com/install-pyenv-ubuntu-debian):
    • sudo apt-get install -y make build-essential libssl-dev zlib1g-dev libbz2-dev \ libreadline-dev libsqlite3-dev wget curl llvm libncurses5-dev libncursesw5-dev \ xz-utils tk-dev libffi-dev liblzma-dev
    • curl -L https://raw.githubusercontent.com/pyenv/pyenv-installer/master/bin/pyenv-installer | bash
    • pyenv install 3.12

  • make git bare repo for source code and add post-receive hook:

    #!/bin/bash
GIT_WORK_TREE=/var/www/hardware.darkfi git checkout -f main
    • make hook executable with chmod +x hooks/post-receive

  • make folder for website at /var/www/hardware.darkfi; create a new git remote pointing to <user>@<server-ip>:www-code.git and then push to this remote (it should copy all files over to /var/www/darkfi-webshop)

  • install nginx and copy sample file to /etc/nginx/site-available/; enable nginx site by making a symbolic link of it to /etc/nginx/site-enabled/: sudo ln -s /etc/nginx/site-available/hardware.darkfi.nginx /etc/nginx/site-enabled/hardware.darkfi.nginx

  • install git-annex:

    • sudo apt-get install neurodebian
    • sudo apt-get update
    • sudo apt install git-annex

  • make new file /etc/systemd/system/darkfi.service and copy sample

  • sudo systemctl enable darkfi, sudo systemctl start darkfi

  • setup git user info for current user:

    • git config --global user.email "you@example.com"
    • git config --global user.name "Your Name"

  • make git bare repo for content:

    • mkdir www-data
    • cd www-data && git init --bare www-data.git
    • git clone www-data.git content
    • cd content && git annex init
    • add post-receive script to ~/www-data/www-data.git/hooks/
    • run chmod +x ~/www-data/www-data.git/hooks/post-receive.

this will synchronize the git checkout folder (at ~/www-data/content) with the git bare repo we use for actual synchronization (at ~/www-data/www-data.git).