ibeam: Problem: 2-Factor Authentication using IBEAM_TWO_FA_HANDLER=EXTERNAL_REQUEST
Hi All! I am attempting to set up my 2FA automated authentication within IBeam and have hit a roadblock I have been frantically trying to debug before asking for your help.
To be quick - here’s the situation.
-
I have set up a server endpoint
http://{cloud_ip:cloud_port}/sms/codethat returns the IBRK authentication code. NOTE The response is NOT wrapped in JSON - just the string of the authenticaton code. Per the wiki, I assume that is what IBeam is expecting. Is that correct? -
I have thoroughly tested my ‘auth_code_server’ and it works as expected: a. When IBeam triggers an authentication SMS from IBRK- the message is caught by the server and correctly parsed and is ready to provide the desired auth code to IBeam via a GET request to the
http://{cloud_ip:cloud_port}/sms/codeendpoint. b. the needed firewall port for the ‘auth_code_server’ is open - and tests have accessed it outside being on localhost. (I have manually logged into my live IBRK account from which the external server catches and saves the authentication code correctly) c. How long does IBeam wait to get that authentication code? And which IBEAM environment variable controls that? -
I dumped all my IBeam variables in an ‘env.list’ file which is the following:
$ cat /root/ibeam_files/env.list
IBEAM_ACCOUNT={my_account_name}
IBEAM_PASSWORD={my_password}
IBEAM_OUTPUTS_DIR=/srv/outputs
IBEAM_REQUEST_RETRIES=5
IBEAM_OAUTH_TIMEOUT=30
IBEAM_REQUEST_TIMEOUT=30
IBEAM_PAGE_LOAD_TIMEOUT=500
IBEAM_MAX_FAILED_AUTH=100
IBEAM_TWO_FA_HANDLER=EXTERNAL_REQUEST
IBEAM_EXTERNAL_REQUEST_URL=http://{cloud_ip:cloud_port}/sms/code
IBEAM_LOG_LEVEL=DEBUG
- When I spin up m IBeam container - the log looks like this (see below):
Specifically:
1. Why doesn’t my
IBEAM_EXTERNAL_REQUEST_URLshow up in my ‘Configuration’ dictionary? (Oddly-IBEAM_TWO_FA_HANDLER=EXTERNAL_REQUESTdoes appear in the dictionary) 2. Should I define each of the environment variables needed forIBEAM_TWO_FA_HANDLER=EXTERNAL_REQUESTexplicitly? I was assumimg the defaults forIBEAM_TWO_FA_HANDLER=EXTERNAL_REQUESTto be set.
My IBeam launch:
$ docker run --rm --name ibeam --env-file /root/ibeam_files/env.list --network tradenet -p 5000:5000 -v /root/ibeam_files/inputs_directory/:/srv/inputs -v /root/ibeam_files/outputs:/srv/outputs:rw voyz/ibeam:0.5.3
The Log:
2024-05-12 01:22:39,029|I| ############ Starting IBeam version 0.5.3 ############
2024-05-12 01:22:39,031|I| Custom conf.yaml found and will be used by the Gateway
2024-05-12 01:22:39,034|I| Secrets source: env
2024-05-12 01:22:39,035|I| Health server started at port=5001
2024-05-12 01:22:39,035|I| Configuration:
{'INPUTS_DIR': '/srv/inputs/', 'OUTPUTS_DIR': '/srv/outputs', 'GATEWAY_DIR': '/srv/clientportal.gw', 'CHROME_DRIVER_PATH': '/usr/bin/chromedriver', 'GATEWAY_STARTUP': 20, 'GATEWAY_PROCESS_MATCH': 'ibgroup.web.core.clientportal.gw.GatewayStart', 'MAINTENANCE_INTERVAL': 60, 'SPAWN_NEW_PROCESSES': False, 'LOG_LEVEL': 'DEBUG', 'LOG_TO_FILE': True, 'LOG_FORMAT': '%(asctime)s|%(levelname)-.1s| %(message)s', 'REQUEST_RETRIES': 5, 'REQUEST_TIMEOUT': 30, 'RESTART_FAILED_SESSIONS': True, 'RESTART_WAIT': 15, 'REAUTHENTICATE_WAIT': 15, 'HEALTH_SERVER_PORT': 5001, 'SECRETS_SOURCE': 'env', 'GCP_SECRETS_URL': None, 'GATEWAY_BASE_URL': 'https://localhost:5000', 'ROUTE_AUTH': '/sso/Login?forwardTo=22&RL=1&ip2loc=on', 'ROUTE_VALIDATE': '/v1/portal/sso/validate', 'ROUTE_REAUTHENTICATE': '/v1/portal/iserver/reauthenticate?force=true', 'ROUTE_INITIALISE': '/v1/api/iserver/auth/ssodh/init', 'ROUTE_AUTH_STATUS': '/v1/api/iserver/auth/status', 'ROUTE_TICKLE': '/v1/api/tickle', 'ROUTE_LOGOUT': '/v1/api/logout', 'USER_NAME_EL': None, 'PASSWORD_EL': 'NAME@@password', 'SUBMIT_EL': 'CSS_SELECTOR@@.btn.btn-lg.btn-primary', 'ERROR_EL': None, 'SUCCESS_EL_TEXT': 'TAG_NAME@@Client login succeeds', 'OAUTH_TIMEOUT': 30, 'PAGE_LOAD_TIMEOUT': 500, 'ERROR_SCREENSHOTS': False, 'MAX_FAILED_AUTH': 100, 'MIN_PRESUBMIT_BUFFER': 5, 'MAX_PRESUBMIT_BUFFER': 30, 'MAX_IMMEDIATE_ATTEMPTS': 10, 'IBKEY_PROMO_EL_CLASS': 'CLASS_NAME@@ibkey-promo-skip', 'AUTHENTICATION_STRATEGY': 'B', 'MAX_STATUS_CHECK_RETRIES': 120, 'MAX_REAUTHENTICATE_RETRIES': 3, 'UI_SCALING': 1.0, 'TWO_FA_EL_ID': 'ID@@twofactbase', 'TWO_FA_NOTIFICATION_EL': 'CLASS_NAME@@login-step-notification', 'TWO_FA_INPUT_EL_ID': 'ID@@chlginput', 'TWO_FA_HANDLER': 'EXTERNAL_REQUEST', 'STRICT_TWO_FA_CODE': True, 'TWO_FA_SELECT_EL_ID': 'ID@@sf_select', 'TWO_FA_SELECT_TARGET': 'IB Key', 'CUSTOM_TWO_FA_HANDLER': 'custom_two_fa_handler.CustomTwoFaHandler'}
2024-05-12 01:22:39,036|I| Gateway not found, starting new one...
2024-05-12 01:22:39,036|I| Note that the Gateway log below may display "Open https://localhost:[PORT] to login" - ignore this command.
2024-05-12 01:22:39,037|I| Starting Gateway as Linux process with params: ['bash', 'bin/run.sh', 'root/conf.yaml']
running
runtime path : root:dist/ibgroup.web.core.iblink.router.clientportal.gw.jar:build/lib/runtime/*
config file : root/conf.yaml
2024-05-12 01:22:39,046|I| Gateway started with pids: [12]
2024-05-12 01:22:39,047|D| GET https://localhost:5000 (unverified)
2024-05-12 01:22:39,049|I| Cannot ping Gateway. Retrying for another 20 seconds
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by io.netty.util.internal.ReflectionUtil (file:/srv/clientportal.gw/build/lib/runtime/netty-common-4.1.15.Final.jar) to constructor java.nio.DirectByteBuffer(long,int)
WARNING: Please consider reporting this to the maintainers of io.netty.util.internal.ReflectionUtil
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
2024-05-12 01:22:40,049|D| GET https://localhost:5000 (unverified)
2024-05-12 01:22:40,052|I| Cannot ping Gateway. Retrying for another 19 seconds
-> mount demo on /demo
Java Version: 11.0.20
****************************************************
version: a27ed42161ad96c53e715ca5c5e3e3fa4cff5262 Mon, 24 Apr 2023 15:41:53 -0400
****************************************************
This is the Client Portal Gateway
for any issues, please contact api@ibkr.com
and include a copy of your logs
****************************************************
https://www.interactivebrokers.com/api/doc.html
****************************************************
Open https://localhost:5000 to login
App demo is available after you login under: https://localhost:5000/demo#/
2024-05-12 01:22:41,052|D| GET https://localhost:5000 (unverified)
2024-05-12 01:22:41,699|I| Gateway connection established
2024-05-12 01:22:41,700|D| POST https://localhost:5000/v1/api/tickle (unverified)
2024-05-12 01:22:41,834|I| NO SESSION Status(running=True, session=False, connected=False, authenticated=False, competing=False, collision=False, session_id=None, server_name=None, server_version=None, expires=None)
2024-05-12 01:22:41,835|I| Authentication strategy: "B"
2024-05-12 01:22:41,835|I| No active sessions, logging in...
2024-05-12 01:22:41,836|I| Loading auth webpage at https://localhost:5000/sso/Login?forwardTo=22&RL=1&ip2loc=on
2024-05-12 01:22:50,482|D| Targets: {'PASSWORD': Target(NAME@@password), 'SUBMIT': Target(CSS_SELECTOR@@.btn.btn-lg.btn-primary), 'SUCCESS': Target(TAG_NAME@@Client login succeeds), 'IBKEY_PROMO': Target(CLASS_NAME@@ibkey-promo-skip), 'TWO_FA': Target(ID@@twofactbase), 'TWO_FA_NOTIFICATION': Target(CLASS_NAME@@login-step-notification), 'TWO_FA_INPUT': Target(ID@@chlginput), 'TWO_FA_SELECT': Target(ID@@sf_select), 'USER_NAME': Target(NAME@@username), 'ERROR': Target(CSS_SELECTOR@@.xyz-errormessage)}
2024-05-12 01:22:50,532|I| Gateway auth webpage loaded
2024-05-12 01:22:50,532|I| Login attempt number 1
2024-05-12 01:22:50,738|D| target: Target(NAME@@username)
2024-05-12 01:22:56,036|I| Submitting the form
2024-05-12 01:23:26,744|E| Timeout reached when waiting for authentication. The website seems to not be loaded correctly. Consider increasing IBEAM_PAGE_LOAD_TIMEOUT.
Website URL: https://localhost:5000/sso/Login?forwardTo=22&RL=1&ip2loc=on
Exception:
File "/srv/ibeam/ibeam_starter.py", line 168, in <module>
success, shutdown, status = client.start_and_authenticate()
File "/srv/ibeam/src/gateway_client.py", line 53, in start_and_authenticate
success, shutdown, status = self.strategy_handler.try_authenticating(request_retries=request_retries)
File "/srv/ibeam/src/handlers/strategy_handler.py", line 85, in try_authenticating
return self._authentication_strategy_B(status, request_retries)
File "/srv/ibeam/src/handlers/strategy_handler.py", line 140, in _authentication_strategy_B
return self._log_in(status)
File "/srv/ibeam/src/handlers/strategy_handler.py", line 151, in _log_in
success, shutdown = self.login_handler.login()
File "/srv/ibeam/src/handlers/login_handler.py", line 462, in login
self.handle_timeout_exception(e, targets, driver, website_version, self.route_auth, self.base_url, self.outputs_dir)
File "/srv/ibeam/src/handlers/login_handler.py", line 443, in login
self.attempt(targets, wait_and_identify_trigger, driver)
File "/srv/ibeam/src/handlers/login_handler.py", line 367, in attempt
trigger, target = self.step_login(targets, wait_and_identify_trigger, driver, self.secrets_handler.account, self.secrets_handler.password, self.secrets_handler.key, self.presubmit_buffer)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/srv/ibeam/src/handlers/login_handler.py", line 188, in step_login
trigger, target = wait_and_identify_trigger(
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/srv/ibeam/src/handlers/login_handler.py", line 67, in _wait_and_identify_trigger
trigger = WebDriverWait(driver, timeout).until(any_of(*expected_conditions))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/venv/lib/python3.11/site-packages/selenium/webdriver/support/wait.py", line 95, in until
raise TimeoutException(message, screen, stacktrace)
<class 'selenium.common.exceptions.TimeoutException'> Message:
2024-05-12 01:23:26,745|I| Cleaning up the resources. Display: <pyvirtualdisplay.display.Display object at 0x7f58aa2ed1d0> | Driver: <selenium.webdriver.chrome.webdriver.WebDriver (session="d14bc5c4713c8fb18d8c0e4df48617f2")>
2024-05-12 01:23:26,814|I| Logging in failed
2024-05-12 01:23:26,819|I| Starting maintenance with interval 60 seconds
2024-05-12 01:24:26,819|I| Maintenance
Here’s my auth_server_code:
from flask import Flask, request, jsonify
import threading
import time
import logging
import json
## all messages at INFO level and above will be captured
logger = logging.getLogger(__name__)
logger.setLevel(logging.INFO)
app = Flask(__name__)
sms_token = None
lock = threading.Lock()
@app.route('/sms/code', methods=['GET'])
def get_sms_code():
global sms_token
try:
logger.info('request to /sms/code')
wait_count_max = 120 # seconds
wait_count = 0
while wait_count < wait_count_max:
with lock:
if sms_token is not None:
break
time.sleep(1)
wait_count += 1
if wait_count % 20 == 0:
logger.info(f'polling for verification code: {wait_count} of {wait_count_max} secs')
if sms_token is None:
logger.info(f'SMS polling interval expired.')
with lock:
if sms_token:
rsp = f"Responding with verificationCode: {sms_token['verificationCode']} obtained at {sms_token['dateTime']}"
logger.info(f'Response:\n {rsp}')
response = sms_token['verificationCode']
sms_token = None
return response
else:
logger.info('No SMS token received.')
return jsonify({'error': 'No SMS token received.'}), 404
except Exception as e:
logger.info(e)
return jsonify({'error': 'Internal Error'}), 500
@app.route('/sms/incoming', methods=['POST'])
def post_sms_incoming():
global sms_token
VERFICATION_CODE_TAG = 'Your requested authentication code'
try:
logger.info(f'posted data:\n{json.dumps(request.json, indent=4)}')
body = request.json.get('content', '')
parts = body.split(':')
if len(parts) == 2:
message_tag, code_part = parts[0].strip(), parts[1].strip()
if message_tag == VERFICATION_CODE_TAG and len(code_part) == 6:
logger.info(f'Parsed Verification Code: {code_part}')
with lock:
sms_token = {
'verificationCode': code_part,
'dateTime': time.strftime('%Y-%m-%dT%H:%M:%S')
}
else:
logger.info(f'Invalid verification code= {body}.')
else:
logger.info(f'Non-verification code SMS detected= {body}.')
except Exception as e:
logger.info(e)
finally:
response = {}
if sms_token: response = sms_token
return jsonify(response), 200
if __name__ == '__main__':
app.run(host='0.0.0.0', port=5051)
Thanks again for all your time, patience and help!
About this issue
- Original URL
- State: closed
- Created 3 years ago
- Comments: 15 (7 by maintainers)
Very helpful. Thanks!
On Thu, May 23, 2024 at 1:54 AM voyz @.***> wrote: