MultiSnark tool (Class/example, incomplete)

Discussions about i2P enabled Bittorrent clients, experience reports and issues
Post Reply
User avatar
cumlord
Posts: 152
Joined: Thu Oct 05, 2023 5:01 pm
Location: Erect, NC
Contact:

MultiSnark tool (Class/example, incomplete)

Post by cumlord »

Python tool for monitoring snark instances and sending stop/start commands. It needs BeautifulSoup for parsing the webpage, only tested on snark+ standalone.

It was made as part of a cli multi-tool for automated torrent creation. the main tool has been crashing after giving it a ssh-port forwarding manager (for managing multi router/snark setups), so i ripped out just the snark parsing/command part

Mainly i wanted to do autoload balancing by starting/stopping the next snark in the list and maybe keep some stats, not sure where the sweet spot is yet.

How to use

If you do:

Code: Select all

snarkurl = ['127.0.0.1:8002', '127.0.0.1:8003']
Then set:

Code: Select all

snarking = Snark(snarkurl)

Code: Select all

snarking.upspeed[0] = upload speed in K/s for snark on port 8002.
other info you can pull that way:
  • upspeed / downspeed
  • peer_connections / dht_peers
  • tunnels_in / tunnels_out
  • color (changes the color of up/downspeed for cli implementation with for use with rich)
  • online (bol)
  • stalled (True if snark is reachable but stopped)
  • nonce
Start/ stop commands:

Bottom has examples for sending start/stop commands to all snarks

Tool
privatebin.i2p link
cpaste.i2p link

Code: Select all

#!/usr/bin/env python3
from bs4 import BeautifulSoup
import requests
import re

###snark urls example
snarkurl = [
'http://127.0.0.1:8001/snark/',
'127.0.0.1:8002/',
'127.0.0.1:8003',
'https://127.0.0.1:8004',
'https://127.0.0.1:8005',
'127.0.0.1:8006',
'127.0.0.1:8007',
'127.0.0.1:8008',
'127.0.0.1:8009',
'127.0.0.1:8010'
]	

class Snark:
	def __init__(self, urls):
		self.urls = urls	
		self.url = []
		self.upspeed = []
		self.downspeed = []
		self.peer_connections = []
		self.dht_peers = []
		self.dest = []
		self.tunnels_in = []
		self.tunnels_out = []
		self.shortened_url = []
		self.color = []
		self.online = []
		self.stalled = []
		self.nonce = []

		def snark_parser(url):
			"""
			Sets values from the parsing functions. Returns in this order:
			url, upspeed, downspeed, peer_connections, dht_peers, dest, tunnels_in, 
			tunnels_out, color, Online (bol), Stalled (bol), nonce

			Keyword arguments:
			url -- snark url given as http://host:port/i2psnark/.ajax/xhr1.html
			"""
			def snark_get_table(url):
				"""
				Uses BeatufiulSoup to get snark table and nonce. Returns rows as
				all <th> and get_nonce from <input> tags

				Keyword arguments:
				url -- snark url given as http://host:port/i2psnark/.ajax/xhr1.html
				"""
				try:
					response = requests.get(url)
					soup = BeautifulSoup(response.text, 'html.parser')
					get_nonce = soup.find('input')
					table = soup.find('tfoot')
					rows = table.find_all('th')
					if response.status_code == 200:
						return rows, get_nonce
					else:
						return False, ''
				except Exception as e:
					return False, ''

			def parse_tag(tag, mod, html_string):
				"""
				Parses the data between html tags, returns value as a string
				If there are modifiers after the tag a space is used, if
				mod is '' no space is used for simple tags.

				Keyword arguments:
				tag -- html tag
				mod -- modifiers after the tag
				html_string -- the html string to be parsed
				"""
				if mod != '':
					space = ' '
				else:
					space = ''
				reg_str = f"<{tag}{space}{mod}>(.*?)</{tag}>"
				res = re.findall(reg_str, html_string)
				return(res)

			def snark_speed_convert_kbs(speed_string):
				"""
				Converts speed string given when parsed from snark as x K/s to a float in K/s.

				Keyword arguments:
				speed_string -- speed string given by snark as x K/s.
				"""
				if speed_string != '':
					x = speed_string.split('\u202f')
					if x[1] == 'K/s':
						speed = float(x[0])
					elif x[1] == 'M/s':
						speed = float(x[0]) * 1000
					else:
						speed = float(x[0]) / 1000
					return(speed)
				else:
					return(0)

			def parse_totals(string):
				"""
				Parse totals in bottom of snark between the span tags.
				Splits with '•' to return a list.

				Keyword arguments:
				string -- html string from snark table.
				"""
				span = string.find_all('span')
				x = str(span[0]).split(' • ')
				return(x)

			rows, get_nonce = snark_get_table(url)
			if rows != False:
				totals = parse_totals(rows[0])
				try:
					nonce = (str(get_nonce).split(' ')[3]).split('"')[1]
					upspeed = (parse_tag('th', 'class="rateUp" title="Total upload speed"', str(rows[5])))
					downspeed = (parse_tag('th', 'class="rateDown" title="Total download speed"', str(rows[3])))
					peer_connections = (totals[2].split(' '))[0]
					dht_peers_convert = ((parse_tag('span', '', totals[3])[0]).split(' '))[0]
					dht_peers = int(dht_peers_convert.replace(',', ''))
					dest = parse_tag('code', '', totals[4])[0]
					tunnels_in_out = (((totals[4].split('</span>'))[0]).split('Tunnels: '))[1]
					tunnels_in = int(((tunnels_in_out.split(' / ')[0]).split(' '))[0])
					tunnels_out = int(((tunnels_in_out.split(' / ')[1]).split(' '))[0])
					upspeed = snark_speed_convert_kbs(upspeed[0])
					downspeed = (snark_speed_convert_kbs(downspeed[0]))
					color = '[green1]'
					return(url, upspeed, downspeed, peer_connections, dht_peers, dest, tunnels_in, tunnels_out, color, True, False, nonce)
				except Exception as e:
					color = '[orange1]'
					return(url, 0, 0, 0, 0, 'STOP', 0, 0, color, False, True, nonce)
			else:
				color = '[red1]'
				return(url, 0, 0, 0, 0, 'OFFL', 0, 0, color, False, False, '')
		
		def get_short_url(snarkurl):
			'''
			Converts urls to the snark console to host:port format.
			Returns host:port and concats to /i2psnark/.ajax/xhr1.html

			Keyword arguments:
			snarkurl -- url to snark instance	
			'''
			host_port = snarkurl
			p = '(?:http.*://)?(?P<host>[^:/ ]+).?(?P<port>[0-9]*).*'
			m = re.search(p,snarkurl)
			host = m.group('host')
			port = m.group('port')
			host_port = f'{host}:{port}'
			fullurl = f"http://{host_port}/i2psnark/.ajax/xhr1.html"
			return host_port, fullurl
		
		for item in snarkurl:
			host_port, fullurl = get_short_url(item)
			snarking = snark_parser(fullurl)
			self.url.append(snarking[0])
			self.upspeed.append(snarking[1])
			self.downspeed.append(snarking[2])
			self.peer_connections.append(snarking[3])
			self.dht_peers.append(snarking[4])
			self.dest.append(snarking[5])
			self.tunnels_in.append(snarking[6])
			self.tunnels_out.append(snarking[7])
			self.shortened_url.append(host_port)
			self.color.append(snarking[8])
			self.online.append(snarking[9])
			self.stalled.append(snarking[10])
			self.nonce.append(snarking[11])

def post_command_snark(snark_shortened_url, action, nonce):
	"""
	Sends post request to snark to start or stop all torrents.

	Keyword arguments:
	snark_shortened_url -- host:port with no http://
	action -- accepts 'start' or 'stop'
	nonce -- nonce for the snark instance
	"""
	if action.lower() == 'start':
		command_key = 'action_StartAll'
		command_value = 'Start All'
	if action.lower() == 'stop':
		command_key = 'action_StopAll'
		command_value = 'Stop All'
	headers = {
		'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8',
		'Accept-Language': 'en-US,en;q=0.5',
		'User-Agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:12.0) Gecko/20100101 Firefox/12.0',
		'Referer': f'http://{snark_shortened_url}/i2psnark/',
		'Origin': f'http://{snark_shortened_url}',
		'Connection': 'keep-alive',
		'Upgrade-Insecure-Requests': '1',
		'Sec-Fetch-Dest': 'iframe',
		'Sec-Fetch-Mode': 'navigate',
		'Sec-Fetch-Site': 'same-origin',
		'Sec-Fetch-User': '?1',
	}	
	data = {
		'nonce': nonce,
		command_key: command_value,
	}
	response = requests.post(f'http://{snark_shortened_url}/i2psnark/_post', headers=headers, data=data)



snarking = Snark(snarkurl)
for i in range(0, len(snarkurl)):
	print(f"{snarking.shortened_url[i]} {snarking.dest[i]} ▲ {int(snarking.upspeed[i])}K/s {snarking.downspeed[i]}K/s ▼ {snarking.peer_connections[i]} peers {snarking.dht_peers[i]} dht peers")

	### start all stopped snarks
	# if snarking.online[i] == True:
	# 	post_command_snark(snarking.shortened_url[i], 'start', snarking.nonce[i])

	### start all paused snarks
	# if snarking.stalled[i] == True:
	# 	post_command_snark(snarking.shortened_url[i], 'start', snarking.nonce[i])

	#### stop all online snarks
	# if snarking.online[i] == True:
	# 	post_command_snark(snarking.shortened_url[i], 'stop', snarking.nonce[i])
test version in the toolbox with rich
http://o7jgnp7bubzdn7mxfqmghn3lzsjtpgkb ... ARKED.jpeg
Last edited by cumlord on Fri Aug 23, 2024 5:07 am, edited 2 times in total.
User avatar
cumlord
Posts: 152
Joined: Thu Oct 05, 2023 5:01 pm
Location: Erect, NC
Contact:

Re: MultiSnark tool (Class/example, incomplete)

Post by cumlord »

Realized i should mention a way to run multiple snark standalones, i thought it'd be a lot more involved for some reason but it's just changing the ports in 3 places, setting the download folder outside of the snark folder, allowing it to check all torrents, then copy and paste, change the ports on those.

This is "safe" to do if you want to open up more tunnels serving the same content, not to download.
do NOT download on snarks sharing the same directory! That will be fucky.

Or you could do this to organize content in different directories.

get snark+ (check http://skank.i2p/):

Code: Select all

eepget "http://skank.i2p/installers/I2P+_2.6.0+_i2psnark-standalone.zip"
Or may need to go to /i2p directory and do ./eepget
or download the torrent.

unzip and cd to i2psnark dir

Code: Select all

sudo unzip I2P+_2.6.0+_i2psnark-standalone.zip && cd i2psnark

EDIT PORTS
Change the 8002 in both areas to 8001 (or whatever you want)

Code: Select all

sudo nano launch-i2psnark.bat
Change the 8002 to 8001.

Code: Select all

sudo nano jetty-i2psnark.xml
OPTIONAL: Disable browser launch on startup, system tray, notification popups. If doing headless no need for this to be enabled, if on gui i'd think it'd get obnoxious

Code: Select all

sudo nano i2psnark-appctx.config
uncomment these things:

Code: Select all

#
# This is for app context configuration of standalone i2psnark.
# Almost all configuration settings are in i2psnark.config.d/i2psnark.config
#
# disable browser launch on startup
routerconsole.browser=/bin/false
# disable browser launch on startup (Windows)
#routerconsole.browser=NUL
# change browser
#routerconsole.browser=firefox
# disable system tray
desktopgui.enabled=false
# disable system tray notification popups
desktopgui.showNotifications=false
OPTIONAL: modify the directory and change your bandwidth settings before launch.

Code: Select all

cd i2psnark.config.d && sudo nano i2psnark.config
Change this to where your torrents are stored. Ensure this is somewhere outside of the snark directory.
i2psnark.dir=/directory/to/torrents

You can always just change it in the webui but it's nice to have it start checking torrents right away.
start it, and when it's done checking your torrents, you can copy things over. Go back to your i2psnark directory.

you can do:

Code: Select all

sudo mv i2psnark i2psnark1 && sudo cp -r i2psnark1 i2psnark2 && sudo cp -r i2psnark1 i2psnark3......
If you're on systemd linux you can run them as services, such as:

Code: Select all

[Unit]
Description=launch snark instance
After=network.target

[Service]
User=i2p
Restart=on-failure
RestartSec=5
ExecStart=/path/to/snark/i2psnark1/./launch-i2psnark

[Install]
WantedBy=multi-user.target
Post Reply