YaCy and I2P

[lgillis]

The P2P search engine YaCy was and is used here by a few operators, but always exclusively without using the special "P2P" feature. This is not ideal for several reasons, just think of the failure of a search engine and its database. The promise of I2P is that anything that is possible on the Internet should be possible here. To make a long story short, anyone who wants to is welcome to help potential future YaCy operators with the initial setup. If you know how to set up the search engine in the P2P network, you are welcome to share your knowledge. In return, you can brag about it for as long as you like.

Think of the possibilities created by a network of failsafe search engines. Users can instantly find much more of the information they came to the network for in the first place. This brings customers and vendors together in a targeted way. The economic aspect is often overlooked, but this is fundamentally wrong. Look at the Tor network, its continued and growing success is based on free trade.

• https://yacy.net/ YaCy is free software for your own search engine
• https://www.yacy.net/operation/yacy-tor/ YaCy and Tor

[kit knox]

I just set it up, and it was a huge pain in the neck. I can go back and make step by step instructions but it'll take a couple of weeks; I'm buried at work at the moment. The current index is a little under 5 million but that's artificially high because of all the reddit front ends, etc. It's all I2P, no clear net or Tor URLs.

But here it is, try it out:

http://1search.i2p/
(B32) http://v366ky2twqpbmlqug6e2n3v43dy5sgvu ... q.b32.i2p/
(AH) http://1search.i2p/?i2paddresshelper=O3 ... AEAAcAAA==

I'm still working on load balancing and multi-homing but the hardware is all enterprise class hardware so it should be responsive.

Let me know what you think,

--Kit

[cumlord]

agree, i remember it being a pain to set up, but it seems like it's pretty much the best option to crawl i2p without relying on the outernet. It's been a while but i think the default crawl settings are (or were) very aggressive, so make sure the settings are respectful so we're not scrapping the handful of i2p sites into oblivion haha.

unfortunately not able to reach your instance

[kit knox]

I was working on it yesterday, it's available again. I need to figure out a few things: it's indexing .css, .js, etc files. I've tried several filters but they don't appear to be affecting the index.

[lgillis]

I need to figure out a few things: it's indexing .css, .js, etc files. I've tried several filters but they don't appear to be affecting the index.

Mh, should the file types mentioned be indexed or not? This is how it looks in the default setting: YaCy Search Interface: Filetype. It is therefore intended that these file types are also indexed.

[kit knox]

You're correct but that's intended for intranet indexing and not website indexing I believe? I don't want it to produce search results that jump straight to JavaScript scripts.

I think I figured it out now, it seems to be working smoothly. I'll continue to tune it and then create a custom front end so it's not so bland.

--Kit