TCP P2P, Event driven!

[twitchyliquid64]

Hi all,

Firstly, considerable credit must go to Kip, as the event driven TCP is primarily based off his UDF.

Objective of this UDF:

To provide dead-easy functionality for internet/LAN applications that do not need/want a server. In simpler terms, an easy to use P2P engine.

If your confused, a simplistic way of thinking of this would be to think of it as a re-write of kip's TCP UDF, that allows a 'client and server in the same script'. =]

PLEASE READ THE ENTIRE POST IF YOU WANT TO UNDERSTAND HOW TO USE THIS.

How it works:

##################

When you start the UDF, it will start listening and accepting incoming connections. The UDF will, in the backround, continue to manage all your connections and initiate any new ones should you use the _P2P_Connect( IP) function. It will also manage the recieving and sending of data.

As a P2P node should, it will route data destined for others, and it will also maintain a healthy number of connections (the number which you define when you start the node).

This entire system is event driven, so you will register your functions with the UDF and the function will get called when the corresponding event happens. See the comments in the UDF for more information.

This works in the same way as Kip's TCP UDF.

Structure:

##################

The first thing you must do is start the Node:

_P2P_Start_Node( Node Identifier, Port, Max peers, Bootstrap mode, Bootstrap max, local/global, IP)

All the Parameters are pretty self expanatory. The Node identifier is the name of your computer in the network. It must not change across sessions and it must be unique. It is used to identify you in your network.

Bootstrapping means autoconnecting. if Bootstrap mode is 1, It will autoconnect you to others until you reach your Bootstrap Max.

Local/Global - Put 0 if working on a LAN, 1 if over internet.

IP - (Optional). You shouldn't need to change this. It is the IP of the listening port.

The second thing is declare your functions. This tells the UDF which functions to call when something happens. EG:

_P2P_Register_Event($P2P_RECEIVE, "my_recieve_function")

So, this will call the function my_recieve_function when data from a peer is recieved. See the comments at the beginning of the UDF for a complete list of Event values, and the parameters for your functions.

Then fire away!

_P2P_Send( $socket, "MY DATA!!)

_P2P_Connect( "127.0.0.1")

_P2P_Broadcast( "My data")

When your done,

_P2P_Stop_Node()

EDIT:

There is also a very powerful system I forgot to mention earlier called long range message routing. Basically, it allows you to send messages in your P2P network to peers you are not connected to, but someone else in the network is. Consider this:

P1 is connected to P2 and P3. P4 is connect to P3. P1 wants to send message to P4.

You would use my function _P2P_Send_Message. It will route your message from P1 -> P3 -> P4 automatically. The speed of processing for three nodes will be approx. 90ms, so it will be relatively fast.

UDF:

P2PTCP.au3

EXAMPLE:

P2Ptest.au3

Not the best example but it will do. Notice that the UDF by default prints to console.

Hope you enjoy it!

Hypoz.

Edited November 16, 2010 by hyperzap

[ludocus]

Sounds great man!

[Yorn]

FAQ:

I can't use #8i8TERMINATE# in my code. Why?

It is a required string of the code itself. This is to prevent TCP runoff that plagues most TCP transmissions, including those of us who were using the event driven TCP script and ran into this problem. Having a terminate request solves this problem of runoff and makes the actual "events" work.

[Zibit]

Well done !

[Zibit]

#include <guicombo.au3> isnt working

[jvanegmond]

I've been going through the code and I have some comments. It seems like you've written this for yourself and created an awesome app around it (looking at the includes in the test), and before posting you decided it needed a short example. In my opinion, the example is far too short and doesn't nearly show the full potential of the UDF. Also, dare I say, that this makes network communication without a server more complicated instead of simpler. It does make working with P2P a lot simpler.

I made a shortlist of what I thought so far.

• _Get_IP() is in the standard AutoIt functions. It's called _GetIP() there and is found in Inet.au3.
• I'd really like to see a longer example. Possibly something with a GUI that runs directly on a standard AutoIt installation. (Maybe a GUI that shows connected clients, all the clients in the network (by IP), realtime traffic, traceroute functionality with #hops. And to top it all off, a network topology of connected subnets.)
• You can allow people to use #8i8TERMINATE# in their message. For an example how that works check out HTTP multipart. (In practice, the header string is generated from the browser name and a random value. For example: --webkit9876987asd765786fg57sdf89fg.)
• For situations like this:

  $arrayslot = Findfreearrayslot()
  if $arrayslot = "error" then ;No more spots for new peers. Connection dropped.

  you can use the @error macro manually with SetError

• Instead of copy pasting

  if $console_out = True then ConsoleWrite

  everywhere I recommend using something like

  Func _DebugWrite($s)
  if $console_out = True then ConsoleWrite($s)
  EndFunc

And the UDF could have used a bit more comments. Certain parts were hard to read through.

Edited October 1, 2010 by Manadar

[twitchyliquid64]

I've been going through the code and I have some comments. It seems like you've written this for yourself and created an awesome app around it (looking at the includes in the test), and before posting you decided it needed a short example. In my opinion, the example is far too short and doesn't nearly show the full potential of the UDF. Also, dare I say, that this makes network communication without a server more complicated instead of simpler. It does make working with P2P a lot simpler.

I made a shortlist of what I thought so far.

• _Get_IP() is in the standard AutoIt functions. It's called _GetIP() there and is found in Inet.au3.
• I'd really like to see a longer example. Possibly something with a GUI that runs directly on a standard AutoIt installation. (Maybe a GUI that shows connected clients, all the clients in the network (by IP), realtime traffic, traceroute functionality with #hops. And to top it all off, a network topology of connected subnets.)
• You can allow people to use #8i8TERMINATE# in their message. For an example how that works check out HTTP multipart. (In practice, the header string is generated from the browser name and a random value. For example: --webkit9876987asd765786fg57sdf89fg.)
• For situations like this:

  $arrayslot = Findfreearrayslot()
  if $arrayslot = "error" then ;No more spots for new peers. Connection dropped.

  you can use the @error macro manually with SetError

• Instead of copy pasting

  if $console_out = True then ConsoleWrite

  everywhere I recommend using something like

  Func _DebugWrite($s)
  if $console_out = True then ConsoleWrite($s)
  EndFunc

And the UDF could have used a bit more comments. Certain parts were hard to read through.

Thankyou for your time.

I will get on to making a better example when I can. I have almost finished my P2P program, which is a massed hi-speed message and IRC based messenger. Once thats done, I will create a better example.

The Get_IP() Function as you said is from the Inet.au3 library. The reason I popped it in was because I thought UDF's were not allowed to have #Include thingies. Then I changed the Function name slightly to avoid collision.

I originally considered using a random number system followed by a delimiter as a header to a message (much like the Message ID system in the long range message functionality). The issue was that this would slow down the delievery and it seemed to be a waste of time. Honestly, Who would use the string #8i8TERMINATE#? A user writing messages would never. I would only be worried of Binary or random data and even then its a probability of (26*2+1)*14 which is a high number.

"

you can use the @error macro manually with SetError

"

What functional advantage would this hold?

"INstead of copying and pasting: if $console_out = True then ConsoleWrite"

Yer, I added the if statment at the last minute in case someone wanted to turn of the Console prints. Its dirty. If will fix it in the next release.

I took a bit more time to make sure this UDF was a bit cleaner than my normal code with aptly named Vars and all that. Looks like it still needs a bit of work

Thanks sooo much. This is the feedback I was looking for. I will get round to it when I can.

Hypoz.

[twitchyliquid64]

#include <guicombo.au3> isnt working

My first guess would be a var clash, and not to mention this was written on an old version of Autoit. Please post your code so I can test more extensively.

[twitchyliquid64]

Guys if any of you who have used this in a program can posts your scripts (if you like), I would like to get some idea of how others use this. Also if anyone wants more functions, let me know.

[lsakizada]

[DELETED BY ME]

Edited February 14, 2011 by lsakizada

[nullschritt]

I am currently considering a P2P solution for my Instant Messaging Client.

However I have some questions about your UDF.

So say each PC authenticates with a web server, and when it does it stores the IP of the computer connecting.

Then say I use your UDF to use the most recent connection as a node. (so each time a client connects, the last client connected is added as a node)

client 1 -> client 2

client 3 -> client 2

client 4 -> client 3

client 5 -> client 4

That your UDF could make connections between all of these nodes, even if one only connects to one other? (so say client 5 could send a message to client 2, even though they never directly connected)

And my last question, if my previous question is true, does your udf automatically connect more nodes when it discovers them (directly or indirectly) [so say if client 4 disconnects, would client 5 automatically connect to client 3 instead, or would I have to add client 3 as a new node manually?]

Sorry if the answers to my question are obvious! I've never used P2P with anything before.

Edit: Also does P2P require port forwarding behind a router? (if so I am sure I could make a non-p2p fallback system for users unable to port forward)

Edited August 30, 2012 by nullschritt

[DeltaRocked]

Hi Hypoz,

I have gone through the UDF and I needed your help in setting up Google App engine.

regards

[twitchyliquid64]

hi,

is it possible to wait 4 days? i will be able to get this working for you after then.

sorry.

[DeltaRocked]

sure.

[nullschritt]

I have a quick question, is the UDF set up so a user can still receive messages, even if they cannot act as a server(they for some reason cannot port forward), will the outbound tcp connections they make to other clients, provide them with data routed to them, or would I have to implement this manually?

[nullschritt]

Not to resurect an old thread, but this is an amazing UDF, I'm supprised more people arn't using it.

There seems to be a bug, if the data sent is too large, the message never arrives at the destination.

Could anyone show an example of how to send larger content(such as images and audio) over a p2p network?

Would it have to be split into packets? If so how would I go about processing the packets on the other side?

[BrewManNH]

nullschritt, you don't have to post this in every TCP related thread you know, we get it, there's a limit. Stop already.

[nullschritt]

This looks great for what I need.

 I need to build a vote system in p2p, Can I do this over WAN?

 Any tips?

I was thinking of setting a sql server on the net to capture ip and port and the sqlite localy to start p2p

Do I need both the port and IP for Nat to transfer the data?

?

HELP!!! lol

Check out my updated version of this script @ '?do=embed' frameborder='0' data-embedContent>>

This system works over WAN or Local Network.

If you wish to use over WAN at least ONE of the P2P nodes needs to have port-forwarding enabled, and this Node's IP address must be somehow communicated to the clients.

The fastest way to facilitate this is to store the IP of any nodes on a web server, and use that web server to give the IP of the available node(s) to the client(s).

A slower but more fool-proof method would be to scan all IP addresses for other P2P nodes (starting with the local network, then scanning all similar IPs, and eventually scanning all IPs on the internet) until a node is found. Though this scanning would have to occur in a separate process, as to not freeze the main thread.

If you still need help, feel free to open a help thread and PM me the link.