get the proxy ip from ngrok

README.md

@@ -1,6 +1,9 @@
 # <b> Google Colab Automator, without Selenium! </b>
 
 ## <b> What is this </b>
+This program creates a botnet to execute arbitrary code using multiple Google Colab VMs.<br>
+
+### <b> Google Colab and Qwiklabs </b>
 Google Colab is a Google service which gives free, temporary, use of virtual machines with powerful NVIDIA Tesla GPUs. Normally this is used to train ML and other AI models, but nothing impedes the use of the service for other purposes, such has password cracking with hashcat or crypto mining (at least in the free version - in Google Colab Pro crypto mining is forbidden).
 The downside is that the account gets temporary blocked when a too andlong intense use is detected.
 Qwiklabs (qwiklabs.com) is a third-party service which gives temporary Google Accounts for training in Google Cloud Shell use. Their first course (https://www.qwiklabs.com/focuses/2794?parent=catalog) is completely free, others require some in-site credits to be purchased.
@@ -8,15 +11,31 @@ Combining the two services, infinite temporary and disposable Google Accounts ca
 <br>
 <br>
 
-## <b> Why no selenium </b>
-Selenium carries some javascript and other stuff which can easily be detected by most websites - GColab and Qwiklabs included - and get the session blocked, especially if captchas are involved (There are workarounds to captchas for selenium - https://github.com/ohyicong/recaptcha_v2_solver - but often the captcha doesn't even get presented to the user if an automated browser session is detected)
-<br>
-<br>
-## <b> The workaround </b>
-Use a plain chrome/firefox session, and automate by programmatically controlling mouse and keyboard. Basic anonimizing of the web session is still needed (spoof ip/mac, change useragent/geolocation/timezone). Actions need to be done at a human speed, or the bot could be recognized as an actual one. This type of use seems to be detected as legit enough to not even display a captcha when starting a Lab.
+## <b> The whole exploit </b>
+1) Prepare a list of multiple Qwiklabs accounts.
+2) Get temp Google account from Qwiklabs (The MAC address has to be spoofed). However, when starting a lab on qwiklabs, sometimes the website presents a captcha and prevents the user from resolving it via audio (common bypass) for the following reasons:
+
+    * Account accessed too many times in a too little timeframe
+    * Tor, common VPN providers and most free proxies are recognized and presented a captcha
+    * Selenium is recognized as an automated browser session
+
+    This problems can easily be bypassed by starting a chrome session with an empty profile, spoofing the MAC but not the IP and automating the browser session by programmatically moving mouse and keyboard. (This is done in <i>get_account_from_qwiklabs.py</i>)
+
+3) Access Google Colab with the newly obtained account, start a Proxy. Google Colab doesn't get recognized as a proxy
+4) Start another script, which is proxed on GColab. This will create and destroy qwiklabs accounts on the need to replace accounts that have been blocked. Restart the colab-proxy session each time to get a new ip, and spoof the mac accordingly
+5) Again, get a new ip from the proxy-colab session. Now head over again to qwiklabs.com, get a new google account and use this to start any notebook. The code will be automatically executed, the needed backend (None, GPU, TPU) can be choosen
+6) Every 45 minutes, get new accounts from qwiklabs to replace the ones currently is use. Always get new accounts while proxed
 <br>
 <br>
 
-## <b> Mouse and Keyboard control </b>
+
+## <b> What to do when some website prevent the use of Selenium </b>
+Selenium carries some javascript and other stuff which can easily be detected by most websites - GColab and Qwiklabs included - and get the session blocked, especially if captchas are involved (There are workarounds to captchas for selenium - https://github.com/ohyicong/recaptcha_v2_solver - but often the  audio captcha doesn't even get presented to the user if an automated browser session is detected)
+<br>
+
+### <b> The workaround: Mouse and Keyboard control</b>
+Use a plain chrome/firefox session, and automate by programmatically controlling mouse and keyboard. Basic anonimizing of the web session is still needed (spoof ip/mac, change useragent/geolocation/timezone). Actions need to be done at a human speed, or the bot could be recognized as an actual one. This type of use seems to be detected as legit enough to not even display a captcha when starting a Lab.<br>
 Mouse and keyboard are controlled using https://pyautogui.readthedocs.io/en/latest/<br>
-For simpicity, and to avoid messing up the OS by clicking on the wrong stuff, both chrome and pyautogui can be started in a separate Xephyr (https://wiki.archlinux.org/title/Xephyr) window, where they can harm no one. This also gives consistent screen coordinates to the elements on the screen: Google chrome will always be fullscreen inside Xephyr. Mouse coordinates are relative to the X server the script is started in. Using a nested X server with Xephyr requires <b>copyq</b> to share the clipboard with the main X server
+For simpicity, and to avoid messing up the OS by clicking on the wrong stuff, both chrome and pyautogui can be started in a separate Xephyr (https://wiki.archlinux.org/title/Xephyr) window, where they can harm no one. This also gives consistent screen coordinates to the elements on the screen: Google chrome will always be fullscreen inside Xephyr. Mouse coordinates are relative to the X server the script is started in. Using a nested X server with Xephyr requires <b>copyq</b> to share the clipboard with the main X server
+<br>
+<br>

browser_manager.py

@@ -0,0 +1,71 @@
+# from proxy_rotator import rotator
+
+# from sudo import run_as_sudo
+import undetected_chromedriver as uc
+uc.install()
+
+from selenium import webdriver
+
+from selenium.webdriver.common.proxy import Proxy, ProxyType
+from selenium.webdriver.common.desired_capabilities import DesiredCapabilities
+from selenium.webdriver.chrome.options import Options
+from selenium.webdriver.support.ui import WebDriverWait
+from selenium.common.exceptions import TimeoutException
+from selenium.webdriver.support import expected_conditions as EC
+import time
+import random
+import requests
+
+
+def clickButton(browser, by, selector, timeout=5):
+    WebDriverWait(browser, timeout).until(EC.element_to_be_clickable((by, selector))).click()
+    time.sleep(1)
+
+def inputText(browser, by, selector, text, timeout=5):
+  element = WebDriverWait(browser, timeout).until(EC.element_to_be_clickable((by, selector)))
+  for i in text:
+    element.send_keys(i)
+    time.sleep(0.25)
+    
+# A lifesaver: https://stackoverflow.com/questions/36141681/does-anybody-know-how-to-identify-shadow-dom-web-elements-using-selenium-webdriv
+def select_shadow_element_by_css_selector(browser, selector):
+  running_script = 'return document.querySelector("%s").shadowRoot' % selector
+  element = browser.execute_script(running_script)
+  return element
+
+# Another lifesaver: https://stackoverflow.com/questions/37384458/how-to-handle-elements-inside-shadow-dom-from-selenium
+# def expand_shadow_element(driver, element):
+#   shadow_root = driver.execute_script('return arguments[0].shadowRoot', element)
+#   return shadow_root
+
+# This variant works on firefox  https://stackoverflow.com/questions/67133483/accessing-shadowroot-via-selenium-in-firefox-returns-javascriptexception-cyclic
+def expand_shadow_element(driver, element):
+    # return a list of elements
+    shadowRoot = driver.execute_script('return arguments[0].shadowRoot.children', element)
+    return shadowRoot
+
+def start_browser(headless=False, proxy=False):
+#   # change mac
+#   # run_as_sudo('macchanger -r wlp3s0')
+
+#   time.sleep(3)
+
+#   #Set Chromedriver Options
+  opts = webdriver.ChromeOptions()
+  if headless is True:
+    options.headless = True
+      
+  opts.add_argument('--enable-javascript') #enabling javascript is needed in order to not get recognized as a bot
+  # opts.add_argument("user-agent={}".format(user_agent))
+  if proxy is not False:
+    opts.add_argument(f'--proxy-server=socks5://'+proxy)
+
+  #Fire up chromedriver
+  chromedriver = webdriver.Chrome(options=opts)
+
+  time.sleep(3)
+  return chromedriver
+
+def quit_browser(driver):
+  driver.delete_all_cookies()
+  driver.quit()

chrome-profile/Default/.com.google.Chrome.1s0BhK

@@ -0,0 +1 @@
+{"net":{"http_server_properties":{"servers":[],"version":5},"network_qualities":{"CAISABiAgICA+P////8B":"4G"}}}

chrome-profile/Default/Local Storage/leveldb/CURRENT

@@ -0,0 +1 @@
+MANIFEST-000001

chrome-profile/Default/Local Storage/leveldb/LOG

@@ -0,0 +1,2 @@
+2021/08/23-17:54:35.446 3 Creating DB /home/emamaker/Documents/Projects/GColabAutomator-NoSelenium/chrome-profile/Default/Local Storage/leveldb since it was missing.
+2021/08/23-17:54:35.522 3 Reusing MANIFEST /home/emamaker/Documents/Projects/GColabAutomator-NoSelenium/chrome-profile/Default/Local Storage/leveldb/MANIFEST-000001

chrome-profile/Default/README

@@ -0,0 +1 @@
+Google Chrome settings and storage represent user-selected preferences and information and MUST not be extracted, overwritten or modified except through Google Chrome defined APIs.

chrome-profile/Default/Secure Preferences

@@ -0,0 +1 @@
+{"protection":{"super_mac":"33F663353631B144EA660B5F809D89BA41CAF954EE5776BE5004BB589CA96BE1"}}

chrome-profile/Default/Site Characteristics Database/CURRENT

@@ -0,0 +1 @@
+MANIFEST-000001

chrome-profile/Default/Site Characteristics Database/LOG

@@ -0,0 +1,2 @@
+2021/08/23-17:54:35.514 ef7 Creating DB /home/emamaker/Documents/Projects/GColabAutomator-NoSelenium/chrome-profile/Default/Site Characteristics Database since it was missing.
+2021/08/23-17:54:35.541 ef7 Reusing MANIFEST /home/emamaker/Documents/Projects/GColabAutomator-NoSelenium/chrome-profile/Default/Site Characteristics Database/MANIFEST-000001

chrome-profile/Default/Sync Data/LevelDB/CURRENT

@@ -0,0 +1 @@
+MANIFEST-000001

chrome-profile/Default/Sync Data/LevelDB/LOG

@@ -0,0 +1,2 @@
+2021/08/23-17:54:35.614 ed9 Creating DB /home/emamaker/Documents/Projects/GColabAutomator-NoSelenium/chrome-profile/Default/Sync Data/LevelDB since it was missing.
+2021/08/23-17:54:35.630 ed9 Reusing MANIFEST /home/emamaker/Documents/Projects/GColabAutomator-NoSelenium/chrome-profile/Default/Sync Data/LevelDB/MANIFEST-000001

chrome-profile/Last Version

@@ -0,0 +1 @@
+92.0.4515.159

chrome-profile/Local State

@@ -0,0 +1 @@
+{"data_use_measurement":{"data_used":{"services":{"background":{},"foreground":{}},"user":{"background":{},"foreground":{}}}},"hardware_acceleration_mode_previous":true,"legacy":{"profile":{"name":{"migrated":true}}},"policy":{"last_statistics_update":"13274207675264769"},"profile":{"info_cache":{"Default":{"avatar_icon":"chrome://theme/IDR_PROFILE_AVATAR_26","background_apps":false,"gaia_id":"","is_consented_primary_account":false,"is_ephemeral":false,"is_guest":false,"is_using_default_avatar":true,"is_using_default_name":true,"managed_user_id":"","name":"Person 1","signin.with_credential_provider":false,"user_name":""}},"last_active_profiles":[],"profile_counts_reported":"13274207675334550"},"profile_network_context_service":{"http_cache_finch_experiment_groups":"None None None"},"shutdown":{"num_processes":0,"num_processes_slow":0,"type":0},"subresource_filter":{"ruleset_version":{"checksum":0,"content":"","format":0}},"ukm":{"persisted_logs":[]},"uninstall_metrics":{"installation_date2":"1629734074"},"user_experience_metrics":{"low_entropy_source3":7700,"pseudo_low_entropy_source":4066,"session_id":0,"stability":{"browser_last_live_timestamp":"13274207675754910","child_process_crash_count":0,"crash_count":0,"exited_cleanly":true,"extension_renderer_crash_count":0,"extension_renderer_failed_launch_count":0,"extension_renderer_launch_count":0,"gpu_crash_count":0,"incomplete_session_end_count":0,"launch_count":1,"page_load_count":0,"renderer_crash_count":0,"renderer_failed_launch_count":0,"renderer_hang_count":0,"renderer_launch_count":0,"session_end_completed":false,"stats_buildtime":"1628878608","stats_version":"92.0.4515.159-64"}},"was":{"restarted":false}}

chrome-profile/SingletonCookie

@@ -0,0 +1 @@
+1401135808698284839

chrome-profile/SingletonLock

@@ -0,0 +1 @@
+emamaker-HP-ENVY-3745

chrome-profile/SingletonSocket

@@ -0,0 +1 @@
+/tmp/.com.google.Chrome.AcrQrN/SingletonSocket

colab_automator.sh

@@ -1,6 +1,13 @@
 #!/bin/bash
 
-#change mac
+# kill -9 $(pidof python)
+
+source venv/bin/activate #switch to virtual env
+
+# install and update requirements
+pip install -r requirements.txt
+
+# change mac
 sudo ip link set wlp3s0 down
 sudo macchanger -r wlp3s0
 sudo ip link set wlp3s0 up
@@ -9,18 +16,15 @@ sudo ip link set wlp3s0 up
 Xephyr -br -ac -noreset -screen 800x600 :1 &
 # Start a copyq server, that will be used to copy the email and password of the obtained account from the clipboard of Xephyr
 DISPLAY=:1 copyq &
-# sleep 5
+
+# Clear chrome profile
+rm -rf ./chrome-profile
 
 #Start chrome
-#--proxy-server=socks5://127.0.0.1:9050
-DISPLAY=:1 /usr/bin/google-chrome-stable --user-data-dir='./chrome-profile' --no-first-run  &
+DISPLAY=:1 /usr/bin/google-chrome-stable --user-data-dir='./chrome-profile' --no-first-run &
 
 #Wait enough to let chrome start up
 sleep 5
 
 #Now start the actual bot, we're in a safe environment
-source venv/bin/activate #switch to virtual env
-DISPLAY=:1 python main.py
-
-# Clear chrome profile
-rm -rf ./chrome-profile
+DISPLAY=:1 python get_account_from_qwiklabs.py

currently_unused/choose_proxy_from_list.py

@@ -0,0 +1,44 @@
+#bash -c 'proxybroker --timeout 5 find --types HTTP HTTPS --lvl High --countries GB ES DE IT FR SW US RU --strict -l 25 --format json --outfile proxies.txt'
+#sleep 5
+# proxy=$(python choose_proxy.py proxies.txt)
+#--proxy-server=socks5://127.0.0.1:9050
+# DISPLAY=:1 all_proxy=true /usr/bin/google-chrome-stable --user-data-dir='./chrome-profile' --no-first-run --proxy-server=http://$proxy  &
+
+import sys
+import json
+import random
+import requests
+
+if len(sys.argv) > 2:
+    print("Too many arguments, only the file listing the proxies is needed", file=sys.stderr) 
+elif len(sys.argv) < 1:
+    print("Please input the json file containing the proxy list", file=sys.stderr)
+else:
+    file = open(sys.argv[1])
+
+    string = ""
+    for line in file.readlines():
+        string += line
+    
+    json_file = json.loads(string)
+    proxies = []
+    for i in json_file:
+        proxies.append(str(i['host']) + ":" + str(i['port']))
+
+    proxy_found = False
+    while proxy_found is False:
+        PROXY = proxies[random.randint(0,len(proxies)-1)]
+        # print("\nTrying to use {} as proxy".format(PROXY))
+        try:
+            r = requests.get('https://google.com', proxies={'https': 'http://{}'.format(PROXY), 'http': 'http://{}'.format(PROXY)}, timeout=3)
+            if r.status_code == 200:
+                # print('\n[+] IP Address used as proxy: ' + PROXY)
+                proxy_found = True
+            else:
+                # print('\nProxy is present but returned status code ' + str(r.status_code))
+                proxies.remove(PROXY)
+        except Exception as e: 
+            # print('Failed with an exception: \n' + str(e))
+            proxies.remove(PROXY)
+
+    print(PROXY, file=sys.stdout)

currently_unused/google_accounts.txt

@@ -0,0 +1 @@
+student-00-3b7b6796b09d@qwiklabs.net:sPgQC5fvG72

get_account_from_qwiklabs.py

@@ -0,0 +1,102 @@
+import gui
+import pyautogui
+import time
+import subprocess
+
+def main():
+    pyautogui.FAILSAFE = False
+    
+    screenWidth, screenHeight = pyautogui.size()
+    ui = gui.Gui(screenWidth, screenHeight)
+
+    print("Screen is {} {}".format(screenWidth, screenHeight))
+
+    print("Accessing course on qwiklabs")
+    # Head over to the course on qwiklabs.com
+    gui.Gui.chrome_gui.click_searchbar()
+    pyautogui.write("qwiklabs.com") 
+    pyautogui.press('/')   
+    pyautogui.write("focuses") 
+    pyautogui.press('/')  
+    pyautogui.write("2794") 
+    pyautogui.press('?')  
+    pyautogui.write("parent")
+    pyautogui.press('=') 
+    pyautogui.write("catalog")
+    pyautogui.press('enter')
+
+    time.sleep(15)
+
+    print("Signing in")
+    pyautogui.moveTo(0,0)
+
+    # Click join button
+    pyautogui.moveTo(ui.qwiklabs_gui.JOIN_BTN[0], ui.qwiklabs_gui.JOIN_BTN[1], 3, pyautogui.easeOutQuad) 
+    pyautogui.click(button='left')
+    time.sleep(5)
+
+    # Move to the end of the page
+    pyautogui.press('pgdn')
+
+    print("Moving from sign up to sign in")
+    # switch to sign in button
+    pyautogui.moveTo(ui.qwiklabs_gui.SIGN_IN_BTN[0], ui.qwiklabs_gui.SIGN_IN_BTN[1], 3, pyautogui.easeOutQuad) 
+    pyautogui.click(button='left')
+    time.sleep(5)
+
+    # switch to email textbox, the rest is done via kbd
+    pyautogui.moveTo(ui.qwiklabs_gui.EMAIL_TEXTBOX[0], ui.qwiklabs_gui.EMAIL_TEXTBOX[1], 3, pyautogui.easeOutQuad) 
+    pyautogui.click(button='left')
+    time.sleep(2)
+
+    print("Inserting credentials")
+    # account;  "mopopa1077@5sword.com", "hellogoodbye"
+    account = ("hemerey688@kibwot.com", "hellogoodbye")
+    username = account[0]
+    password = account[1]
+    first_part = username.split('@')[0]
+    second_part = username.split('@')[1]
+
+    pyautogui.press('@')
+    pyautogui.press('left')
+    pyautogui.write(first_part, interval=0.2) 
+    pyautogui.press('right')
+    pyautogui.write(second_part, interval=0.2)
+    pyautogui.press('\t')
+    pyautogui.write(password, interval=0.2)
+    pyautogui.press('enter')
+
+    time.sleep(10)
+
+
+    # Start course
+    time.sleep(5)
+    print("Starting the course")
+    pyautogui.moveTo(ui.qwiklabs_gui.STARTLAB_BTN[0], ui.qwiklabs_gui.STARTLAB_BTN[1], 3, pyautogui.easeOutQuad)
+    pyautogui.click()
+
+    # Accept captcha - sometimes they don't even ask for actual verification
+    # time.sleep(5)
+    # print("Accepting captcha (not needed really)")
+    # pyautogui.moveTo(ui.qwiklabs_gui.CAPTCHA_BTN[0], ui.qwiklabs_gui.CAPTCHA_BTN[1], 3, pyautogui.easeOutQuad)
+    # pyautogui.click()
+
+
+    bashCommand = "copyq clipboard"
+    # Copy username to clipboard
+    time.sleep(20)
+    print("Copying email")
+    pyautogui.moveTo(ui.qwiklabs_gui.EMAIL_COPY_BTN[0], ui.qwiklabs_gui.EMAIL_COPY_BTN[1], 3, pyautogui.easeOutQuad)
+    pyautogui.click()
+    g_email = subprocess.check_output(['bash','-c', bashCommand]).decode('utf-8')
+
+    # Copy password to clipboard
+    time.sleep(5)
+    print("Copying password")
+    pyautogui.moveTo(ui.qwiklabs_gui.PASSWORD_COPY_BTN[0], ui.qwiklabs_gui.PASSWORD_COPY_BTN[1], 3, pyautogui.easeOutQuad)
+    pyautogui.click()
+    g_password = subprocess.check_output(['bash','-c', bashCommand]).decode('utf-8')
+
+    print(g_email, g_password)
+
+main()

gui.py

@@ -2,6 +2,7 @@
 This classes holds the coordinates for all the elements that need to be clicked on the screen.
 All the coordinates are kept as percentages of the screen resolution and are recalculated at runtime
 '''
+import pyautogui
 
 class Gui:
     def __init__(self, width, height):
@@ -14,6 +15,14 @@ class Chrome(Gui):
     def __init__(self, width, height):
         Chrome.SEARCH_BAR=width*Chrome.SEARCH_BAR_PERCENTAGE[0], height*Chrome.SEARCH_BAR_PERCENTAGE[1]
 
+    def click_searchbar(self):
+        pyautogui.moveTo(Chrome.SEARCH_BAR[0], Chrome.SEARCH_BAR[1])
+        pyautogui.click()
+        pyautogui.keyDown('ctrl')
+        pyautogui.press('a')
+        pyautogui.keyUp('ctrl')
+
+
 class Qwiklabs(Gui):
     JOIN_BTN_PERCENTAGE = (0.775, 0.183)
     SIGN_IN_BTN_PERCENTAGE = (0.2125, 0.73)

main.py

@@ -1,109 +1,6 @@
-import gui
-import pyautogui
-import time
-import subprocess
+import ngrok
 
-def main():
-    pyautogui.FAILSAFE = False
-    
-    screenWidth, screenHeight = pyautogui.size()
-    ui = gui.Gui(screenWidth, screenHeight)
-
-    print("Screen is {} {}".format(screenWidth, screenHeight))
-
-    print("Accessing course on qwiklabs")
-    # Head over to the course on qwiklabs.com
-    pyautogui.press('f6')  
-    pyautogui.write("qwiklabs.com") 
-    pyautogui.press('/')   
-    pyautogui.write("focuses") 
-    pyautogui.press('/')  
-    pyautogui.write("2794") 
-    pyautogui.press('?')  
-    pyautogui.write("parent")
-    pyautogui.press('=') 
-    pyautogui.write("catalog")
-    pyautogui.press('enter')
-
-    time.sleep(5)
-
-    print("Signing in")
-    pyautogui.moveTo(0,0)
-
-    # Click join button
-    pyautogui.moveTo(ui.qwiklabs_gui.JOIN_BTN[0], ui.qwiklabs_gui.JOIN_BTN[1], 3, pyautogui.easeOutQuad) 
-    pyautogui.click(button='left')
-    time.sleep(5)
-
-    # Move to the end of the page
-    pyautogui.press('pgdn')
-
-    print("Moving from sign up to sign in")
-    # switch to sign in button
-    pyautogui.moveTo(ui.qwiklabs_gui.SIGN_IN_BTN[0], ui.qwiklabs_gui.SIGN_IN_BTN[1], 3, pyautogui.easeOutQuad) 
-    pyautogui.click(button='left')
-    time.sleep(5)
-
-    # switch to email textbox, the rest is done via kbd
-    pyautogui.moveTo(ui.qwiklabs_gui.EMAIL_TEXTBOX[0], ui.qwiklabs_gui.EMAIL_TEXTBOX[1], 3, pyautogui.easeOutQuad) 
-    pyautogui.click(button='left')
-    time.sleep(2)
-
-    print("Inserting credentials")
-    # account;  "mopopa1077@5sword.com", "hellogoodbye"
-    account = ("mopopa1077@5sword.com", "hellogoodbye")
-    username = account[0]
-    password = account[1]
-    first_part = username.split('@')[0]
-    second_part = username.split('@')[1]
-
-    pyautogui.press('@')
-    pyautogui.press('left')
-    pyautogui.write(first_part, interval=0.2) 
-    pyautogui.press('right')
-    pyautogui.write(second_part, interval=0.2)
-    pyautogui.press('\t')
-    pyautogui.write(password, interval=0.2)
-    pyautogui.press('enter')
-
-    time.sleep(10)
-
-
-    # Start course
-    time.sleep(5)
-    print("Starting the course")
-    pyautogui.moveTo(ui.qwiklabs_gui.STARTLAB_BTN[0], ui.qwiklabs_gui.STARTLAB_BTN[1], 3, pyautogui.easeOutQuad)
-    pyautogui.click()
-
-    # Accept captcha - sometimes they don't even ask for actual verification
-    # time.sleep(5)
-    # print("Accepting captcha (not needed really)")
-    # pyautogui.moveTo(ui.qwiklabs_gui.CAPTCHA_BTN[0], ui.qwiklabs_gui.CAPTCHA_BTN[1], 3, pyautogui.easeOutQuad)
-    # pyautogui.click()
-
-
-    bashCommand = "copyq clipboard"
-    # Copy username to clipboard
-    time.sleep(20)
-    print("Copying email")
-    pyautogui.moveTo(ui.qwiklabs_gui.EMAIL_COPY_BTN[0], ui.qwiklabs_gui.EMAIL_COPY_BTN[1], 3, pyautogui.easeOutQuad)
-    pyautogui.click()
-    g_email = subprocess.check_output(['bash','-c', bashCommand]).decode('utf-8')
-
-    # Copy password to clipboard
-    time.sleep(5)
-    print("Copying password")
-    pyautogui.moveTo(ui.qwiklabs_gui.PASSWORD_COPY_BTN[0], ui.qwiklabs_gui.PASSWORD_COPY_BTN[1], 3, pyautogui.easeOutQuad)
-    pyautogui.click()
-    g_password = subprocess.check_output(['bash','-c', bashCommand]).decode('utf-8')
-
-    print(g_email, g_password)
-
-    
-
-
-
-    # while True:
-    #     print(pyautogui.position())
-
-main()  
+n = ngrok.Ngrok()
+n.start_browser()
+n.access_ngrok()
+print(n.get_proxy_ip())

ngrok.py

@@ -0,0 +1,37 @@
+# access ngrok with selenium, get the first ip/port combination
+# https://dashboard.ngrok.com/endpoints/status
+# automatically accepts key and inputs password
+# sshpass -p hellogoodbye ssh -D 1337 -q -C -N root@ip -p port  -o "StrictHostKeyChecking no"
+
+import browser_manager
+from selenium.webdriver.common.by import By
+import time
+
+class Ngrok:
+    def __init__(self):
+        pass
+
+    def start_browser(self):
+        self.driver = browser_manager.start_browser()
+
+    def close_browser(self):
+        browser_manager.quit_browser(self.driver)
+
+    def access_ngrok(self):
+        self.driver.get('https://dashboard.ngrok.com/endpoints/status')
+        time.sleep(5)
+        browser_manager.inputText(self.driver, By.CSS_SELECTOR, '#email', 'giangillo.rossi1@gmail.com')
+        time.sleep(1)
+        browser_manager.inputText(self.driver, By.CSS_SELECTOR, '#password', 'emamaker02')
+        time.sleep(1)
+        browser_manager.clickButton(self.driver, By.XPATH, '/html/body/div[2]/div/section/main/div/div/div[2]/div[1]/div/form/div[3]/div/div/div/button')
+        time.sleep(5)
+
+    # Assumes access_ngrok has already been called
+    def get_proxy_ip(self):
+        self.driver.get('https://dashboard.ngrok.com/endpoints/status')
+        time.sleep(10) #takes a bit to load, just in case
+        proxy_line = self.driver.find_element_by_xpath('/html/body/div[2]/div/section/section/main/div/div/div/main/div[2]/div/div/div/div/div/table/tbody/tr[2]/td[3]').text
+        # strip down the tcp:// part at the start, it's not needed
+        proxy = proxy_line[6:]
+        return proxy

requirements.txt

@@ -1,2 +1,4 @@
 pyautogui
-pyperclip
+requests
+undetected-chromedriver
+selenium==3.141.0

start_colab_machine.py

@@ -0,0 +1 @@
+import subprocess