R & D of Multiple Semi-Autonomous Systems

Location: Adelaide, SA or Brisbane, QLD

Note: project must be undertaken at one of the two locations listed above

Duration: 5 months

Keywords: Python, Matlab, R, Machine learning, multidiscipline, team-focused, problem solving

This internship is only available to Australian Citizens due to funding eligibility requirements. All applicants who are not Australian Citizens will be automatically deemed ineligible.

Project Background

Autonomous systems will play an increasing role in defence, but they will need to work alongside and under the command of human war fighters.  These systems will need to quickly learn and adapt to changing situations and needs.  DST Group is working toward this vision by undertaking R&D in machine learning and Human-On-The-Loop control of multiple semi-autonomous systems.

Reinforcement learning is an approach where an agent adapts its behaviour to maximise the reward it obtains from its environment.  Google Deep Mind recently demonstrated the ability for a deep network combined with reinforcement learning to learn a range of computer games for the Atari. Of the 50 games attempted around half performed better than a human. However, as first-“person” games none involved control of multiple agents.

The learning problem becomes more complicated if the environment contains other agents who can also learn and adapt. The agent will need to adjust to new behaviour, and should ideally be prepared for the most likely possible behaviours from other agents.  The agent may be able to improve its performance by estimating the knowledge and intentions of the other agents.  The agent might be able to recognise when it lacks understanding, and seek help from a human or another artificial agent. DST is interested in exploring these kinds of issues in multi-agent reinforcement learning and human autonomy teaming.

Research to be Conducted

This research project will apply adversarial learning, and assess the strength of the resulting agents against humans and hand-coded agents.

DST Group are developing a simple game based on a defence scenario to investigate multi-agent learning and high-level command for human autonomy teaming. In the game, the blue side must protect some installations against attack from the red side.  Each side controls several units.  Variants of the game will have these units controlled by high level commands (carry out an expanding-square search centered at x, y) or low-level commands (adjust direction by -5 degrees), either by a single agent per side, or with each unit controlled by a separate agent. The Open AI Gym environment will be used for some of these game configurations.

The project will set up reinforcement learning agents to play this game against one another, and see what level of skill they are able to achieve.  Simple human user interfaces, and hand-coded players will be created for comparison.  Different problems posed by variants of the game will be explored.

Depending on time, progress and the student’s interests, other aspects of the broad problem discussed above may also be investigated.

Skills Required

For this project, we are looking for PhD students with:

  • Computer programming. Python will be used, but is easy to learn for experienced programmers. Matlab or R would also be useful background though not a requirement.
  • Knowledge of Machine Learning, especially Reinforcement Learning.
  • A good team player in a multidisciplinary setting, with a general interest in learning and contributing to a team research effort.
  • Willingness to broaden skills and knowledge beyond current academic field.

The intern does not need to already have expertise in machine learning, so long as they have a strong relevant background and are willing and able to quickly learn what is needed.  Although the role would ideally suit a computer science student, students of statistics, neuroscience, theoretical economics, electrical engineering, robotics, control theory and other related disciplines are encouraged to consider applying.

Expected Outcomes

The research is expected to inform DST’s future work on defence applications of autonomy. The final report will make recommendations on further research, and the application of adversarial learning and related techniques in mixed human autonomy teams.

Co-authorship of a conference paper with DST Group researchers would also be a likely outcome.

Additional Details

The intern will receive $3,000 per month of the internship, usually in the form of stipend payments.

It is expected that the intern will primarily undertake this research project during regular business hours, spending at least 50% (this % is negotiable) of their time on-site with the industry partner. The intern will be expected to maintain contact with their academic mentor throughout the internship either through face-to-face or phone meetings as appropriate.

The intern and their academic mentor will have the opportunity to negotiate the project’s scope, milestones and timeline during the project planning stage.

To participate in the AMSI Intern program, all applicants must satisfy the following criteria:

  • Be a PhD student currently enrolled at an Australian University.
  • PhD candidature must be confirmed.
  • Applicants must have the written approval of their Principal Supervisor to undertake the internship. This approval must be submitted at the time of application.
  • Have Australian Citizenship
  • Internships are also subject to any requirements stipulated by the student’s and the academic mentor’s university.
Applications Close

21 November 2017

Reference:
INT – 0364