SQLite performance

[Skysnake]

; open SQLite with a specific local file
Local $sLocalSQLiteDll = @ScriptDir & "\sqlite3.dll" ; to be change to an existing .dll to have no error

Local $sSQliteDll = _SQLite_Startup($sLocalSQLiteDll, False, 1)
    If @error Then
        MsgBox($MB_SYSTEMMODAL, "SQLite Error", "SQLite3.dll Can't be Loaded!" & @CRLF & @CRLF & _
                $sLocalSQLiteDll & " Not FOUND in @SystemDir, @WindowsDir, @ScriptDir or @WorkingDir")
        Exit -1
    EndIf

Downloaded most recent, set to Usex64=N and restarted. Added the code above

Output

>"C:\Program Files (x86)\AutoIt3\SciTE\..\AutoIt3.exe" "C:\Program Files (x86)\AutoIt3\SciTE\AutoIt3Wrapper\AutoIt3Wrapper.au3" /run /prod /ErrorStdOut /in "C:\AutoIt\SQLite_benchmark\SQLite_speedtest.au3" /UserParams    
+>17:13:14 Starting AutoIt3Wrapper v.17.224.935.0 SciTE v.3.7.3.0   Keyboard:00000409  OS:WIN_10/  CPU:X64 OS:X64  Environment(Language:0409)  CodePage:0  utf8.auto.check:4
+>         SciTEDir => C:\Program Files (x86)\AutoIt3\SciTE   UserDir => C:\Users\user\AppData\Local\AutoIt v3\SciTE\AutoIt3Wrapper   SCITE_USERHOME => C:\Users\user\AppData\Local\AutoIt v3\SciTE 
>Running AU3Check (3.3.14.2)  from:C:\Program Files (x86)\AutoIt3  input:C:\AutoIt\SQLite_benchmark\SQLite_speedtest.au3
+>17:13:14 AU3Check ended.rc:0
>Running:(3.3.14.2):C:\Program Files (x86)\AutoIt3\autoit3.exe "C:\AutoIt\SQLite_benchmark\SQLite_speedtest.au3"    
--> Press Ctrl+Alt+Break to Restart or Ctrl+Break to Stop
in: Row: 3648   Col: 10 Time: 1069.31
ex: Row: 0  Col: 0  Time: 45.76
in: Row: 3648   Col: 10 Time: 1043.83
ex: Row: 0  Col: 0  Time: 37.52
in: Row: 3648   Col: 10 Time: 1041.53
ex: Row: 0  Col: 0  Time: 36.25
in: Row: 3648   Col: 10 Time: 1040.96
ex: Row: 0  Col: 0  Time: 37.85
in: Row: 3648   Col: 10 Time: 1042.00
ex: Row: 0  Col: 0  Time: 35.88
+>17:13:20 AutoIt3.exe ended.rc:0
+>17:13:20 AutoIt3Wrapper Finished.
>Exit code: 0    Time: 6.013

 

[LarsJ]

You'll not get anything faster than this:

expandcollapse popup

#AutoIt3Wrapper_Au3Check_Parameters=-d -w 1 -w 2 -w 3 -w 4 -w 5 -w 6

;#AutoIt3Wrapper_UseX64=y

#include <GUIConstants.au3>
#include <WindowsConstants.au3>
#include <GuiListView.au3>
#include <SQLite.au3>

Opt( "MustDeclareVars", 1 )

Global $pTable, $iCols, $hLV

Example()

Func Example()
  _SQLite_Startup()
  _SQLite_Open( "random_data.db" )

  Local $iRows
  Local $sSQL = "SELECT * FROM RandomData;"
  _SQLite_GetTableEx( -1, $sSQL, $pTable, $iRows, $iCols )
  $pTable += $iCols * ( @AutoItX64 ? 8 : 4 )

  GUICreate( "Virtual ListView", 1650, 800 )

  Local $idLV = GUICtrlCreateListView( "", 10, 10, 1650-20, 800-20, $LVS_OWNERDATA, BitOR( $WS_EX_CLIENTEDGE, $LVS_EX_DOUBLEBUFFER, $LVS_EX_FULLROWSELECT ) )
  $hLV = GUICtrlGetHandle( $idLV ) ;                                Virtual listview                          Reduces flicker
  For $i = 0 To $iCols - 1
    _GUICtrlListView_AddColumn( $idLV, "Col" & $i, 160 )
  Next

  GUICtrlSendMsg( $idLV, $LVM_SETITEMCOUNT, $iRows, 0 )
  GUIRegisterMsg( $WM_NOTIFY, "WM_NOTIFY" )
  GUISetState( @SW_SHOW )

  While 1
    Switch GUIGetMsg()
      Case $GUI_EVENT_CLOSE
        ExitLoop
    EndSwitch
  WEnd

  $pTable -= $iCols * ( @AutoItX64 ? 8 : 4 )
  _SQLite_FreeTable( $pTable )
  _SQLite_Close()
  _SQLite_Shutdown()
EndFunc

Func WM_NOTIFY( $hWnd, $iMsg, $wParam, $lParam )
  Static $iPtrSize = @AutoItX64 ? 8 : 4, $hDLL = DllOpen( "kernel32.dll" ), $aDisplay[100][$iCols], $iFrom, $iTo
  Static $tText = DllStructCreate( "wchar[4094]" ), $pText = DllStructGetPtr( $tText )
  Static $tagNMLVCACHEHINT = $tagNMHDR & ";int iFrom;int iTo"

  Local $tNMHDR, $hWndFrom, $iCode
  $tNMHDR = DllStructCreate( $tagNMHDR, $lParam )
  $hWndFrom = HWnd( DllStructGetData( $tNMHDR, "hWndFrom" ) )
  $iCode = DllStructGetData( $tNMHDR, "Code" )

  Switch $hWndFrom
    Case $hLV
      Switch $iCode
        Case $LVN_GETDISPINFOW
          Local $tNMLVDISPINFO = DllStructCreate( $tagNMLVDISPINFO, $lParam )
          If Not BitAND( DllStructGetData( $tNMLVDISPINFO, "Mask" ), 1 ) Then Return ; 1 = $LVIF_TEXT
          Local $iItem = DllStructGetData( $tNMLVDISPINFO, "Item" ) - $iFrom
          If $iItem < 0 Or $iItem > $iTo - $iFrom Then Return
          Local $iSubItem = DllStructGetData($tNMLVDISPINFO,"SubItem")
          Local $sItem = $aDisplay[$iItem][$iSubItem]
          Local $l = StringLen( $sItem )
          DllStructSetData( $tText, 1, $sItem )
          DllStructSetData( $tNMLVDISPINFO, "Text", $pText )
          DllStructSetData( $tNMLVDISPINFO, "TextMax", ( $l <= 4094 ? $l : 4094 ) )
          Return

        Case $LVN_ODCACHEHINT
          Local $tNMLVCACHEHINT = DllStructCreate( $tagNMLVCACHEHINT, $lParam )
          $iFrom = DllStructGetData( $tNMLVCACHEHINT, "iFrom" )
          $iTo = DllStructGetData( $tNMLVCACHEHINT, "iTo" )
          Local $pFrom, $pStr, $iLen, $tWstr
          For $i = $iFrom To $iTo
            $pFrom = $pTable + $i * $iCols * $iPtrSize
            For $j = 0 To $iCols - 1
              $pStr = DllStructGetData( DllStructCreate( "ptr", $pFrom + $j * $iPtrSize ), 1 )
              $iLen = DllCall( $hDLL, "int", "MultiByteToWideChar", "uint", 65001, "dword", 0, "ptr", $pStr, "int", -1, "ptr", 0, "int", 0 )[0]
              $tWstr = DllStructCreate( "wchar[" & $iLen & "]" )
              DllCall( $hDLL, "int", "MultiByteToWideChar", "uint", 65001, "dword", 0, "ptr", $pStr, "int", -1, "struct*", $tWstr, "int", $iLen )
              $aDisplay[$i-$iFrom][$j] = DllStructGetData( $tWstr, 1 )
            Next
          Next
          Return
      EndSwitch
  EndSwitch

  Return $GUI_RUNDEFMSG
  #forceref $hWnd, $iMsg, $wParam
EndFunc

Func _SQLite_GetTableEx( $hDB, $sSQL, ByRef $pTable, ByRef $iRows, ByRef $iCols )
  If __SQLite_hChk($hDB, 2) Then Return SetError(@error, 0, $SQLITE_MISUSE)
  Local $tSQL8 = __SQLite_StringToUtf8Struct($sSQL)
  If @error Then Return SetError(3, @error, 0)
  Local $avRval = DllCall($__g_hDll_SQLite, "int:cdecl", "sqlite3_get_table", _
    "ptr", $hDB, _       ; An open database
    "struct*", $tSQL8, _ ; SQL to be executed
    "ptr*", 0, _         ; Results of the query
    "int*", 0, _         ; Number of result rows
    "int*", 0, _         ; Number of result columns
    "long*", 0)          ; Error msg written here
  If @error Then Return SetError(1, @error, $SQLITE_MISUSE) ; DllCall error
  $pTable = $avRval[3]
  $iRows = $avRval[4]
  $iCols = $avRval[5]
EndFunc

Func _SQLite_FreeTable( $pTable )
  DllCall($__g_hDll_SQLite, "int:cdecl", "sqlite3_free_table", "ptr", $pTable)
  If @error Then Return SetError(1, @error, $SQLITE_MISUSE) ; DllCall error
EndFunc

 

[HurleyShanabarger]

That is amazing - how the hell did you come up with this? That is giving me possibilities I wasn't even thinking about.

Just to get a better understanding:

1. It is possible to load multiple databases and perform multiple queries, as long as they are handling with there own pointers
2. as i want to "split" data from one table to different listview I can handle that also within the WM_NOTIFY function

[jchd]

49 minutes ago, HurleyShanabarger said:

It is possible to load multiple databases and perform multiple queries, as long as they are handling with there own pointers

Of course that would be possible, but you can also ATTACH many databases to the open DB and conveniently use SQL to query them all (or part of) at once in a single query.

@LarsJ,

Very fast indeed. One couple (minor) things: NULLs display as 0 and blobs are interpreted as UTF8.

[HurleyShanabarger]

Okay, i made a version where I have three listview filled with data and that more or less dynamically. An global array is holding the pointer, listview IDs, etc. and in the WM_Notify event I am accesing the data to draw the active listview. No comes the complicated part: I want to have the listviews synchronized for scrolling: I know how to do that for a NON virtual listview, but as the data is drawn dynamically to the listview I am lost.

expandcollapse popup

#Region ;**** Directives created by AutoIt3Wrapper_GUI ****
    #AutoIt3Wrapper_UseX64=y
    #Tidy_Parameters=/reel /ri
#EndRegion ;**** Directives created by AutoIt3Wrapper_GUI ****

#include <GUIConstants.au3>

#include <WindowsConstants.au3>
#include <GuiListView.au3>
#include <SQLite.au3>

Global Enum $e_iRecipe, $e_iMaster, $e_iSister, $e_iMaxRow
Global Enum $e_eDB, $e_pTable, $e_iCols, $e_iRows, $e_hLV, $e_iLV, $e_SQL, $e_iMaxCol
Global $g_aHandle[$e_iMaxRow][$e_iMaxCol]
Global $g_iLV_Width = 250
;0.. handle DB
;1.. pointer Data
;2.. columns
;3.. rows
;4.. handle ListView
;5.. identifier listview
;6.. query
Example()

Func Example()
    GUICreate("Virtual ListView", 1650, 800)

    _SQLite_Startup("FileInstall\sqlite3_x64.dll")
    $g_aHandle[$e_iRecipe][$e_eDB] = _SQLite_Open("random_data.db")
    $g_aHandle[$e_iMaster][$e_eDB] = $g_aHandle[$e_iRecipe][$e_eDB]
    $g_aHandle[$e_iSister][$e_eDB] = $g_aHandle[$e_iRecipe][$e_eDB]

    $g_aHandle[$e_iRecipe][$e_SQL] = 'SELECT * FROM RandomData WHERE "Column_1" <> "";'
    $g_aHandle[$e_iMaster][$e_SQL] = 'SELECT * FROM RandomData WHERE "Column_3" <> "";'
    $g_aHandle[$e_iSister][$e_SQL] = 'SELECT * FROM RandomData WHERE "Column_4" <> "";'

    For $l_iLV = 0 To $e_iMaxRow - 1
        _SQLite_GetTableEx($g_aHandle[$l_iLV][$e_eDB], $g_aHandle[$l_iLV][$e_SQL], $g_aHandle[$l_iLV][$e_pTable], $g_aHandle[$l_iLV][$e_iRows], $g_aHandle[$l_iLV][$e_iCols])
        $g_aHandle[$l_iLV][$e_pTable] += $g_aHandle[$l_iLV][$e_iCols] * (@AutoItX64 ? 8 : 4)
        $g_aHandle[$l_iLV][$e_iLV] = GUICtrlCreateListView("", ($l_iLV * ($g_iLV_Width + 10)) + 10, 50, $g_iLV_Width, 400, $LVS_OWNERDATA)
        GUICtrlSetStyle($g_aHandle[$l_iLV][$e_iLV], BitOR($GUI_SS_DEFAULT_LISTVIEW, $LVS_REPORT, $WS_TABSTOP, $WS_VISIBLE, $WS_CHILD, $WS_CLIPSIBLINGS), BitOR($WS_EX_CLIENTEDGE, $LVS_EX_DOUBLEBUFFER, $LVS_EX_FULLROWSELECT, $LVS_EX_GRIDLINES))
        $g_aHandle[$l_iLV][$e_hLV] = GUICtrlGetHandle($g_aHandle[$l_iLV][$e_iLV])
        For $i = 0 To $g_aHandle[$l_iLV][$e_iCols] - 1
            _GUICtrlListView_AddColumn($g_aHandle[$l_iLV][$e_iLV], "Col" & $i, 160)
        Next
        GUICtrlSendMsg($g_aHandle[$l_iLV][$e_iLV], $LVM_SETITEMCOUNT, $g_aHandle[$l_iLV][$e_iRows], 0)
    Next

    GUIRegisterMsg($WM_NOTIFY, "WM_NOTIFY")

    GUISetState(@SW_SHOW)
    While 1
        Switch GUIGetMsg()
            Case $GUI_EVENT_CLOSE
                ExitLoop
        EndSwitch
    WEnd

    For $l_iLV = 0 To $e_iMaxRow - 1
        $g_aHandle[$l_iLV][$e_pTable] -= $g_aHandle[$l_iLV][$e_iCols] * (@AutoItX64 ? 8 : 4)
        _SQLite_FreeTable($g_aHandle[$l_iLV][$e_pTable])
        _SQLite_Close($g_aHandle[$l_iLV][$e_eDB])
    Next

    _SQLite_Shutdown()
EndFunc   ;==>Example

Func _GUI_msg_Generic($f_lParam, $f_iIndx)
    Local Static $s_iPtrSize = @AutoItX64 ? 8 : 4
    Local Static $s_hDLL = DllOpen("kernel32.dll"), $l_iFrom, $l_iTo
    Local $l_aDisplay

    Local Static $s_aDisplay_Recipe[100][$g_aHandle[$e_iRecipe][$e_iCols]]
    Local Static $s_aDisplay_Master[100][$g_aHandle[$e_iMaster][$e_iCols]]
    Local Static $s_aDisplay_Sister[100][$g_aHandle[$e_iSister][$e_iCols]]

    If $f_iIndx = 0 Then $l_aDisplay = $s_aDisplay_Recipe
    If $f_iIndx = 1 Then $l_aDisplay = $s_aDisplay_Master
    If $f_iIndx = 2 Then $l_aDisplay = $s_aDisplay_Sister

    Local Static $s_tText = DllStructCreate("wchar[4094]"), $s_pText = DllStructGetPtr($s_tText)
    Local Static $s_tagNMLVCACHEHINT = $tagNMHDR & ";int iFrom;int iTo"

    Local $l_tNMHDR, $l_hWnd, $l_iCode
    $l_tNMHDR = DllStructCreate($tagNMHDR, $f_lParam)
    $l_hWnd = HWnd(DllStructGetData($l_tNMHDR, "hWndFrom"))
    $l_iCode = DllStructGetData($l_tNMHDR, "Code")

    Switch $l_iCode
        Case $LVN_GETDISPINFOW
            Local $l_tNMLVDISPINFO = DllStructCreate($tagNMLVDISPINFO, $f_lParam)
            If Not BitAND(DllStructGetData($l_tNMLVDISPINFO, "Mask"), 1) Then Return
            Local $l_iItem = DllStructGetData($l_tNMLVDISPINFO, "Item") - $l_iFrom
            If $l_iItem < 0 Or $l_iItem > $l_iTo - $l_iFrom Then Return
            Local $l_iSubItem = DllStructGetData($l_tNMLVDISPINFO, "SubItem")
            Local $l_sItem = $l_aDisplay[$l_iItem][$l_iSubItem]
            Local $l_iItemLen = StringLen($l_sItem)
            DllStructSetData($s_tText, 1, $l_sItem)
            DllStructSetData($l_tNMLVDISPINFO, "Text", $s_pText)
            DllStructSetData($l_tNMLVDISPINFO, "TextMax", ($l_iItemLen <= 4094 ? $l_iItemLen : 4094))
            Return
        Case $LVN_ODCACHEHINT
            Local $tNMLVCACHEHINT = DllStructCreate($s_tagNMLVCACHEHINT, $f_lParam)
            $l_iFrom = DllStructGetData($tNMLVCACHEHINT, "iFrom")
            $l_iTo = DllStructGetData($tNMLVCACHEHINT, "iTo")
            Local $l_pFrom, $l_pStr, $l_iLen, $l_tWstr
            Local $l_iRow, $l_iCol
            For $l_iRow = $l_iFrom To $l_iTo
                $l_pFrom = $g_aHandle[$f_iIndx][$e_pTable] + $l_iRow * $g_aHandle[$f_iIndx][$e_iCols] * $s_iPtrSize
                For $l_iCol = 0 To $g_aHandle[$f_iIndx][$e_iCols] - 1
                    $l_pStr = DllStructGetData(DllStructCreate("ptr", $l_pFrom + $l_iCol * $s_iPtrSize), 1)
                    $l_iLen = DllCall($s_hDLL, "int", "MultiByteToWideChar", "uint", 65001, "dword", 0, "ptr", $l_pStr, "int", -1, "ptr", 0, "int", 0)[0]
                    $l_tWstr = DllStructCreate("wchar[" & $l_iLen & "]")
                    DllCall($s_hDLL, "int", "MultiByteToWideChar", "uint", 65001, "dword", 0, "ptr", $l_pStr, "int", -1, "struct*", $l_tWstr, "int", $l_iLen)
                    $l_aDisplay[$l_iRow - $l_iFrom][$l_iCol] = DllStructGetData($l_tWstr, 1)
                Next
            Next
            If $f_iIndx = 0 Then $s_aDisplay_Recipe = $l_aDisplay
            If $f_iIndx = 1 Then $s_aDisplay_Master = $l_aDisplay
            If $f_iIndx = 2 Then $s_aDisplay_Sister = $l_aDisplay
            Return
    EndSwitch

    Return $GUI_RUNDEFMSG
EndFunc   ;==>_GUI_msg_Generic

Func _GUICtrlListView_SyncScroll($f_hWnd_Src, $f_hWnd_Trg)
    If $f_hWnd_Src = $f_hWnd_Trg Then Return
    Local Static $s_iDeltaY = 14
    Local $l_iLines = _GUICtrlListView_GetTopIndex($f_hWnd_Src) - _GUICtrlListView_GetTopIndex($f_hWnd_Trg)
    _GUICtrlListView_Scroll($f_hWnd_Trg, 0, $l_iLines * $s_iDeltaY)
    Return $l_iLines
EndFunc   ;==>_GUICtrlListView_SyncScroll

Func _SQLite_FreeTable($f_pTable)
    DllCall($__g_hDll_SQLite, "int:cdecl", "sqlite3_free_table", "ptr", $f_pTable)
    If @error Then Return SetError(1, @error, $SQLITE_MISUSE) ; DllCall error
EndFunc   ;==>_SQLite_FreeTable

Func _SQLite_GetTableEx($f_hDB, $f_sSQL, ByRef $b_pTable, ByRef $b_iRows, ByRef $b_iCols)
    If __SQLite_hChk($f_hDB, 2) Then Return SetError(@error, 0, $SQLITE_MISUSE)
    Local $l_tSQL = __SQLite_StringToUtf8Struct($f_sSQL)
    If @error Then Return SetError(3, @error, 0)
    Local $l_Rslt = DllCall($__g_hDll_SQLite, "int:cdecl", "sqlite3_get_table", "ptr", $f_hDB, "struct*", $l_tSQL, "ptr*", 0, "int*", 0, "int*", 0, "long*", 0)
    If @error Then Return SetError(1, @error, $SQLITE_MISUSE)
    $b_pTable = $l_Rslt[3]
    $b_iRows = $l_Rslt[4]
    $b_iCols = $l_Rslt[5]
EndFunc   ;==>_SQLite_GetTableEx

Func WM_NOTIFY($f_hWnd, $f_Msg, $f_wParam, $f_lParam)
    Local $l_tNMHDR = DllStructCreate($tagNMHDR, $f_lParam)
    Local $l_hWnd = HWnd(DllStructGetData($l_tNMHDR, "hWndFrom"))
    Local $l_iIndx = -1
    Switch $l_hWnd
        Case $g_aHandle[$e_iRecipe][$e_hLV]
            $l_iIndx = $e_iRecipe
        Case $g_aHandle[$e_iMaster][$e_hLV]
            $l_iIndx = $e_iMaster
        Case $g_aHandle[$e_iSister][$e_hLV]
            $l_iIndx = $e_iSister
    EndSwitch
    If $l_iIndx > -1 Then
        $_lv_hReturn = _GUI_msg_Generic($f_lParam, $l_iIndx)
        _GUICtrlListView_SyncScroll($g_aHandle[$l_iIndx][$e_hLV], $g_aHandle[$e_iRecipe][$e_hLV])
        _GUICtrlListView_SyncScroll($g_aHandle[$l_iIndx][$e_hLV], $g_aHandle[$e_iMaster][$e_hLV])
        _GUICtrlListView_SyncScroll($g_aHandle[$l_iIndx][$e_hLV], $g_aHandle[$e_iSister][$e_hLV])

        If $_lv_hReturn = $GUI_RUNDEFMSG Then
            Return $GUI_RUNDEFMSG
        Else
            Return
        EndIf

    EndIf
    Return $GUI_RUNDEFMSG
EndFunc   ;==>WM_NOTIFY

 

[LarsJ]

HurleyShanabarger, I've used sqlite3_get_table here in post 25 and I've used virtual listviews here.

I read your other thread yesterday and that's the reason why I came up with the solution in post 22. This implementation of sqlite3_get_table where data is extracted and displayed in the listview directly from the C-array should only be used in relation to virtual listviews.

There is no point in using the C-array to create an AutoIt array. Conversion of the UTF8 strings to wide character strings will take nearly as long time as using _SQLite_GetTable2d. Since blob data is not supported as is stated in the documentation for sqlite3_get_table and also stated by jchd in post 24 it's not worth it.

If only ANSI support is sufficient, extracting data from the C-array is about 4 times as fast as using _SQLite_GetTable2d. In post 25 I'm using pure ANSI data.

There is also no point in filling out a conventional listview with the data from the C-array. Again it'll take nearly as long time as using _SQLite_GetTable2d.

The implementation of sqlite3_get_table should only be used in virtual listviews. And blob data should not be included. It usually does not make sense to display blob data in the cells of a listview.

random_data.db is a pretty small database. The technique can also be used for large databases with millions of rows. Then sqlite3_get_table should be executed as part of the code in the $LVN_ODCACHEHINT section. But then there are other things to be considered. There is a brief discussion in this post. Even for very large databases the virtual listview will show up instantly.

Your requirements in the other thread and in post 23 can easily be implemented.

 

jchd. True, which is also stated in the documentation for sqlite3_get_table. NULLs can be handled with a few lines of additional code.

[HurleyShanabarger]

Lars, I guess you might have been misunderstanding my approach. I am not converting the C-Array (thanks for giving it a name) to an Autoit array. I am going to have multiple virtual list views in one script and therefore simply build an AutoIt Array that contains the pointer to the C-Array, the list view handle etc. 

I have been testing you solution with a database that has 1 GB and I am still impressed with the amazing speed but even more with your professional explanations - I thought that I somehow was reaching some limits with AutoIt when it comes to speed, but you tool that away completely.

I found solutions for the request from this #23 and the other post and will post my solution when I am back home. 

 

Edited September 15, 2017 by HurleyShanabarger

[LarsJ]

I'm well aware that you are not converting the C-Array to an AutoIt array. If you did that the code would not be fast.

From post 22 you can get the impression that you can simply replace _SQLite_GetTable2d with _SQLite_GetTableEx and then suddenly get a function that is 100 times faster. But this is definitely not the case. I just wanted to stress that for you and for everybody else.

_SQLite_GetTableEx is only faster if you can live with that primitive C-array as the function returns. In case of a virtual listview we can live with the C-array. And then the function is about 100 times faster.

For most AutoIt users the C-array is completely useless. You have to convert it to a usual AutoIt array. But that conversion takes a long time. And then you can just as well use _SQLite_GetTable2d from the start.

_SQLite_GetTableEx is in no way the answer to all SQLite performance issues. In relation to virtual listviews it's extremely efficient. But I don't really see any other utilizations.

[HurleyShanabarger]

Understood!

Is there an SQLite forum that you recommend as I have some questions about a way to perform a query and that is not really AutoIt related.

[jchd]

Go ahead, even if it isn't 100% AutoIt. The rationale being that there are enough low-experience AutoIt users who could benefit from various SQL[ite] use cases with possible issues and solutions exposed.