Skip to content
This repository was archived by the owner on Nov 23, 2023. It is now read-only.

Better support for ES running on OpenShift #1

Open
caruccio wants to merge 7 commits into
base: master
Choose a base branch
from
Open

Better support for ES running on OpenShift #1

Show file tree
Hide file tree
Changes from 6 commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
1ccbea9
Fix logs dir
caruccio Nov 18, 2014
0fd74f7
User defined ES port
caruccio Nov 18, 2014
9ad4a71
Instructions on install
caruccio Nov 18, 2014
5e88d68
Update README.md
caruccio Nov 18, 2014
fbcb9c6
Better install message
caruccio Nov 18, 2014
97a2ee8
remove extra comma
caruccio Nov 18, 2014
8de1238
Fix default ES port
caruccio Nov 19, 2014
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,9 +8,11 @@ This is a simple cartridge for making Logstash easily available in your applicat
These environment variables are used when configuring Logstash:

* **`OPENSHIFT_LOGSTASH_ES_HOST`**: URL of the Elasticsearch cluster to log to. Required.
* **`OPENSHIFT_LOGSTASH_ES_PORT`**: Port Elasticsearch is running. Optional, default=9200.
* **`OPENSHIFT_LOGSTASH_ES_USERNAME`**: Username to connect as. Optional.
* **`OPENSHIFT_LOGSTASH_ES_PASSWORD`**: Password to connect with. Optional.

If your Elasticsearch is running under OpenShift itself, set environment variable `OPENSHIFT_LOGSTASH_ES_PORT` to specify the port it's waitings connections on.

@alexbrasetvik alexbrasetvik Dec 12, 2014

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It need not really be running on OpenShift to have a different port, though. A port config makes sense, though. :)


## Installation

Expand Down
21 changes: 20 additions & 1 deletion bin/install
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,22 @@
#!/bin/bash

curl -s https://download.elasticsearch.org/logstash/logstash/logstash-1.4.1.tar.gz | tar xz
source $OPENSHIFT_CARTRIDGE_SDK_BASH

if ! [ ${OPENSHIFT_LOGSTASH_ES_HOST} ]; then
cat >&2 <<-EOF
Missing env var OPENSHIFT_LOGSTASH_ES_HOST.
Please execute the following command and try again:

\$ rhc env set OPENSHIFT_LOGSTASH_ES_HOST=<your.elasticsearch.hostname> -a $OPENSHIFT_APP_NAME -n $OPENSHIFT_NAMESPACE
EOF
exit 1
fi

if ! [ ${OPENSHIFT_LOGSTASH_ES_PORT} ]; then
client_result "Env var OPENSHIFT_LOGSTASH_ES_PORT was not found."
client_result "If your Elasticsearch instance is running under OpenShift, chances are it is bound to port 80 instead 9200."

@alexbrasetvik alexbrasetvik Dec 12, 2014

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why would it prefer port 80?

@paulRbr paulRbr Dec 12, 2014

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Check out https://
https://github.com/getupcloud/openshift-cartridge-elasticsearchgithub.com
https://github.com/getupcloud/openshift-cartridge-elasticsearch/
https://github.com/getupcloud/openshift-cartridge-elasticsearchgetupcloud
https://github.com/getupcloud/openshift-cartridge-elasticsearch/
https://github.com/getupcloud/openshift-cartridge-elasticsearchopenshift-
https://github.com/getupcloud/openshift-cartridge-elasticsearchcartridge-
https://github.com/getupcloud/openshift-cartridge-elasticsearch
elasticsearch
https://github.com/getupcloud/openshift-cartridge-elasticsearch

ES is listening on HTTP in that cartridge.

But I agree ES doesn't need to be running on OpenShift.

In bin/install
#1 (diff)
:

\ No newline at end of file
+source $OPENSHIFT_CARTRIDGE_SDK_BASH
+
+if ! [ ${OPENSHIFT_LOGSTASH_ES_HOST} ]; then

  • cat >&2 <<-EOF
  • Missing env var OPENSHIFT_LOGSTASH_ES_HOST.

  • Please execute the following command and try again:

  •   \$ rhc env set OPENSHIFT_LOGSTASH_ES_HOST=<your.elasticsearch.hostname> -a $OPENSHIFT_APP_NAME -n $OPENSHIFT_NAMESPACE
  • EOF
  • exit 1
    +fi
    +
    +if ! [ ${OPENSHIFT_LOGSTASH_ES_PORT} ]; then
  • client_result "Env var OPENSHIFT_LOGSTASH_ES_PORT was not found."
  • client_result "If your Elasticsearch instance is running under OpenShift, chances are it is bound to port 80 instead 9200."

Why would it prefer port 80?

—
Reply to this email directly or view it on GitHub
https://github.com/foundit/openshift-logstash-cartridge/pull/1/files#r21764358
.

@caruccio caruccio Dec 13, 2014

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Port 80 is the publicly exposed port. In fact I doubt it allows any other external port.
Since this is an openshift cartridge, makes sense for me to sticky with openshift defaults, assuming user is running it's ES inside openshift too.

client_result "If this is the case, please run the following command and restart your app:"
client_result " \$ rhc env set OPENSHIFT_LOGSTASH_ES_PORT=80 -a $OPENSHIFT_APP_NAME -n $OPENSHIFT_NAMESPACE"
fi

curl -s https://download.elasticsearch.org/logstash/logstash/logstash-1.4.1.tar.gz | tar xz
3 changes: 2 additions & 1 deletion conf/logstash.conf.erb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
input {

file {
path => "<%= ENV['OPENSHIFT_LOG_DIR'] %>*.log"
path => "<%= ENV['OPENSHIFT_PRIMARY_CARTRIDGE_DIR'] %>/logs/*"

@alexbrasetvik alexbrasetvik Dec 12, 2014

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This environment variable is not documented here, and it seems OPENSHIFT_LOG_DIR should be preferred? https://developers.openshift.com/en/managing-environment-variables.html#logging-variables

@caruccio caruccio Dec 13, 2014

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, I'm running OpenShift release-3, back when OPENSHIFT_LOG_DIR didn't existed yet. Maybe a better approach is to look on both dirs:

path => [ "<%= ENV['OPENSHIFT_LOG_DIR'] %>*.log", "<%= ENV['OPENSHIFT_PRIMARY_CARTRIDGE_DIR'] %>/logs/*" ]

Please note that old log files may not be using that .log extension. Apache, for instance, writes to {access|error}_log. IMO, a simples /* does a better job.

tags => ["<%= ENV['OPENSHIFT_APP_NAME'] %>", "<%= ENV['OPENSHIFT_GEAR_NAME'] %>", "<%= ENV['OPENSHIFT_NAMESPACE'] %>"]
}

Expand All @@ -13,6 +13,7 @@ output {

elasticsearch_http {
host => "<%= ENV['OPENSHIFT_LOGSTASH_ES_HOST'] %>"
port => "<%= ENV['OPENSHIFT_LOGSTASH_ES_PORT'] || 9300 %>"

@paulRbr paulRbr Nov 19, 2014

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@caruccio defaults to 9200 right?

@caruccio caruccio Nov 19, 2014

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm a typo-o-matic ;) Fixed.

<% if ENV['OPENSHIFT_LOGSTASH_ES_USER'] %>
user => "<%= ENV['OPENSHIFT_LOGSTASH_ES_USER'] %>"
<% end %>
Expand Down