Search the Community

From MiniMax to Machine Learning ... Tic Tac Toe is a good game for studying AI algorithm because it's simple! I use Tabular Q Learning to implement this game, Every time a game finished, it will use the Q function to update the score of each steps it played. Q(S,A) = Q(S,A) + α ∗ (γ ∗ maxaQ(S′,a) − Q(S,A)) S being the current state, A the current action, S′ the state after doing A, α being the learning rate, γ being the discount factor, and maxaQ(S′,a) the highest Q value of any move in the next state S′, i.e. the Q value of the best move in the following state. It's funny to see that it plays better and better. That's why people were charmed by Machine Learning! Thank you! Download: tic_tac_toe.zip

Hello, I'm building whit mine limited coding know-how a AI in Autoit.... Just because..... why not It's not doing what i tought it would do. I hope somebody could help me whit this script? so far : -building an learning grid --> AI needs to guess the label in the grin whit only the X and Y value. -Building an array filled whit random values as weights. -quess the label -learn - if quess not the same as the label go change the weights At this point (the changing of the weights) I've some strange result and hope somebody could point me in the right direction I think that the problem is in the formula for changing the weights. PS. I'm also open for good coding practice I'm learning coding as i go #AutoIt3Wrapper_Au3Check_Parameters=-d -w 1 -w 2 -w 3 -w 4 -w 5 -w 6 -w 7 #include <MsgBoxConstants.au3> #include <Array.au3> build_grid_list() build_neurons() _ArrayDisplay($gridlist,"$gridlist 0","",32) _ArrayDisplay($neurons,"$neurons 0","",32) For $i = 1 To UBound($gridlist,1) - 1 If $gridlist[$i][3] = 1 Then go_learn($neurons[1][0],$gridlist[$i][0]) EndIf Next _ArrayDisplay($gridlist,"$gridlist 1","",32) _ArrayDisplay($neurons,"q$neurons 1","",32) Func go_learn($neuron_ID,$gridlist_ID) Local $test = guess($gridlist_ID , $neuron_ID ) For $i = 1 To 5000 If $test <> $gridlist[$gridlist_ID][4] Then change_wieght($neuron_ID , $gridlist_ID ) $test = guess($gridlist_ID , $neuron_ID ) Else ExitLoop EndIf Next EndFunc Func guess($gridlist_ID , $neuron_ID = 1 ,$raw = 0);, $loop = 0 ) Local $temp = 0 ;~ For $i = 1 To ubound($neurons,1) -1 ; input1 * W1 + input2 * W2 + Bias(1) * W3 $temp = $gridlist[$gridlist_ID][1] * $neurons[$neuron_ID][1] + $gridlist[$gridlist_ID][2] * $neurons[$neuron_ID][2] + 1 * $neurons[$neuron_ID][3] ;activate (sign) If $temp >= 0 Then $neurons[$neuron_ID][6] = 1 Else $neurons[$neuron_ID][6] = -1 EndIf $gridlist[$gridlist_ID][5] = $neurons[$neuron_ID][6] $gridlist[$gridlist_ID][6] = $gridlist[$gridlist_ID][6] + 1 $neurons[$neuron_ID][9] = $neurons[$neuron_ID][9] + 1 If $raw = 0 Then Return $neurons[$neuron_ID][6] Else Return $temp EndIf EndFunc Func build_neurons($needed_neurons = 10 ) Global $neurons[$needed_neurons + 1][10] $neurons[0][0] = "id" $neurons[0][1] = "wieght1" $neurons[0][2] = "wieght2" $neurons[0][3] = "wieght3" $neurons[0][4] = "input1" $neurons[0][5] = "input2" $neurons[0][6] = "output" $neurons[0][7] = "tweak_counter" $neurons[0][8] = "not_tweak_counter" $neurons[0][9] = "quess_counter" Local $temp = 1 For $i = 1 To $needed_neurons; -1 $neurons[$i][0] = $i ;id Do ;zero_check output wieght1 $temp = Random(-1 , 1) Until $temp <> 0 $neurons[$i][1] = $temp ;wieght1 Do ;zero_check output wieght2 $temp = Random(-1 , 1) Until $temp <> 0 $neurons[$i][2] = $temp ;wieght2 Do ;zero_check output wieght3 $temp = Random(-1 , 1) Until $temp <> 0 $neurons[$i][3] = $temp ;wieght3 $neurons[$i][4] = 0 ;input1 $neurons[$i][5] = 0 ;input2 Do ;zero_check output $temp = Random(-1 , 1 , 1) Until $temp <> 0 $neurons[$i][6] = $temp ;output +1 / -1 ;~ $neurons[$i][6] = 0 ;output $neurons[$i][7] = 0 ;tweak_counter $neurons[$i][8] = 0 ;not_tweak_counter $neurons[$i][9] = 0 ;quess_counter Next EndFunc Func build_grid_list($grid_x = 10 ,$grid_y = 10 ) Global $gridlist[($grid_x * $grid_y) + 1 ][7] Local $counter = 1 $gridlist[0][0] = "ID" $gridlist[0][1] = "X" $gridlist[0][2] = "Y" $gridlist[0][3] = "Active" $gridlist[0][4] = "Label" $gridlist[0][5] = "quessed" $gridlist[0][6] = "quessed_counter" For $x = 0 to $grid_x - 1 For $y = 0 to $grid_y - 1 $gridlist[$counter][0] = $counter $gridlist[$counter][1] = $x $gridlist[$counter][2] = $y If Random(-1 , 1) >= 0 Then $gridlist[$counter][3] = 0 Else $gridlist[$counter][3] = 1 EndIf If $x > $y Then $gridlist[$counter][4] = 1 Else $gridlist[$counter][4] = -1 EndIf $gridlist[$counter][5] = -99 $gridlist[$counter][6] = 0 $counter = $counter + 1 Next Next EndFunc Func change_wieght($neuron_id , $grid_id );, $W1 , $W2 ) ;W1 = W1 + ^W1 (some change in W1) ;^W = err(known) * input ($neurons[$id][3] = "output") * learningrate ;$neurons[$id][1] = $neurons[$id][1] + "wieght1" Local $iReturn = False ; Desired | Quess | Error ; -1 -1 0 ; -1 +1 -2 ; +1 -1 +2 ; +1 +1 0 Local $error = $gridlist[$grid_id][4] - $neurons[$neuron_id][6] If $error <> 0 Then Local $learningrate = 0.1 Local $str_len1 = StringLen($neurons[$neuron_id][1]) Local $str_len2 = StringLen($neurons[$neuron_id][2]) Local $str_len3 = StringLen($neurons[$neuron_id][3]) Local $dif_weights1 = $error * $neurons[$neuron_id][1] * $learningrate Local $dif_weights2 = $error * $neurons[$neuron_id][2] * $learningrate Local $dif_weights3 = $error * $neurons[$neuron_id][3] * $learningrate Local $verschil1 = StringLeft($dif_weights1,$str_len1) Local $verschil2 = StringLeft($dif_weights2,$str_len2) Local $verschil3 = StringLeft($dif_weights3,$str_len3) Local $new_wieght1 = $neurons[$neuron_id][1] + $verschil1 Local $new_wieght2 = $neurons[$neuron_id][2] + $verschil2 Local $new_wieght3 = $neurons[$neuron_id][3] + $verschil3 $neurons[$neuron_id][1] = StringLeft($new_wieght1,$str_len1) $neurons[$neuron_id][2] = StringLeft($new_wieght2,$str_len2) $neurons[$neuron_id][3] = StringLeft($new_wieght3,$str_len3) $neurons[$neuron_id][7] = $neurons[$neuron_id][7] + 1 ;"counter" $iReturn = False Else $neurons[$neuron_id][8] = $neurons[$neuron_id][8] + 1 ;"not counter" $iReturn = True EndIf Return $iReturn EndFunc thanks in advanced.

Hi guys, I just finished a Connect 4 game by using MiniMax with Alpha Beta Pruning. I haven't written a program for a long time, but writing an AI program is always funny! I have to learn how the algorithm works and try to optimize the code to run faster. Let's play and have fun! Oops, I lost the game ... Thanks guys! Download: Connect 4.zip

Hi everyone i am creating a speech recognition pc assistant,could any one recommend any ideas regarding what i should include in my project like playing music,telling time and date etc.Thanks in advance.Screenshot of the program gui included as below. Rec013_001.bmp Rec013_002.bmp Rec013_004.bmp