How to get number of worker, cores, ram from HDI cluster

Here is my scenario. I am creating HDI cluster and installing my custom application using ARM template.

I need following values to be configured for my application using Shell script. Installing my application using CustomScript option in ARM template.

Number of worker nodes

Number of cores per worker node

RAM per worker node

RAM per head node

Number of cores per worker node

Answer

You could use Ambari REST API to get number of worker nodes.

PASSWORD=<>
CLUSTERNAME=<>
#Worker nodes
  curl -u admin:$PASSWORD -sS -G "https://$CLUSTERNAME.azurehdinsight.net/api/v1/clusters/$CLUSTERNAME/services/HDFS/components/DATANODE">worker.txt
cat worker.txt |grep total_count|awk -F: '{print $2}'|sed 's/,//g'

RAM per worker node

Do you mean the VM’s max RAM? If yes, every worker node VM should have the same RAM. VM’s RAM and cores are decided to VM’s size. More information please refer to this link. If you want to achieve it with script. I suggest you could write a configure file, such as

Standard_DS1_v2 1   3.5

Standard_DS2_v2 2   7

You could get core and MEM by using awk, it is easy. Such is an example.

mem=`cat configure.txt|grep "Standard_DS1_v2"|awk '{print $3}`