I am struggling getting certain parts of my ELK stack setup but surprisingly, setting up FileBeat to forward logs from IIS in App Services worked first time!

Filebeat is a lightweight exe that can do some very basic log parsing and forwarding, either directly to ElasticSearch or more likely via Logstash, which is a much heavier weight and scalable application that can perform various parsing and modifications of messages before they go into ElasticSearch. In this case, IIS logs should be modified in LogStash to give them more useful metadata.

Logstash is too big and resource hungry to use on App Services (unless it was installed centrally somewhere but that would likely not work well) but fortunately, FileBeat measures in at about 30MB expanded and 8MB in a zip file which is easily small enough for App Services.

The steps are straight-forward and this worked for me with v6.1.1 of filebeat for Windows.


  1. Download filebeat and extract the contents to a folder somewhere to edit
  2. You can delete the install and uninstall as a service PS scripts as well as the reference yml file to save a few KB!
  3. Add a file called run.cmd which includes the command line .\filebeat.exe -e
  4. It is recommended that you test filebeat locally first to ensure your pipeline is working before you bring Azure into the mix but if you already know that works, you can skip that step.
  5. Edit filebeat.yml using the code as shown at the bottom (keep other stuff if you know you need it). The only specific bit for App Services is the log path.
  6. ZIP the contents of your extracted folder by selecting all files and folders in the directory that contains filebeat.exe and choosing Send to compressed (zipped) folder. Do NOT do this on the parent directory, otherwise the zip will include the parent directory at the top level. Call it something like webjob.zip
  7. In the Azure portal, select the App Service you want to use, choose Web Jobs and the + button to add a new one.
  8. Choose a name, select Continuous as a type and select the ZIP file you created, choose multi-instance if these needs to run on every instance of your app (it probably will need to unless you are testing) and press OK.
  9. It will run immediately so make sure that your endpoint is running correctly
Note: The path used on App Services works on my current setup but I cannot tell whether this drive letter and path is guaranteed not to change! I don't currently know how to query the logs directory using e.g. an environment variable.


#=========================== Filebeat prospectors =============================

filebeat.prospectors:

- type: log

  # Change to true to enable this prospector configuration.
  enabled: true

  # Paths that should be crawled and fetched. Glob based paths.
  paths:
    - D:\home\LogFiles\http\RawLogs\*.log

  # Exclude lines. A list of regular expressions to match. It drops the lines that are
  # matching any regular expression from the list.
  exclude_lines: ['^#']

#============================= Filebeat modules ===============================

filebeat.config.modules:
  # Glob pattern for configuration loading
  path: ${path.config}/modules.d/*.yml

  # Set to true to enable config reloading
  reload.enabled: false

#==================== Elasticsearch template setting ==========================

setup.template.settings:
  index.number_of_shards: 1
  #index.codec: best_compression
  #_source.enabled: false

#----------------------------- Logstash output --------------------------------
output.logstash:
  # The Logstash hosts
  hosts: ["logstash.example.com:5044"]