Redis caching with Spring Boot

Hi,

A few example on how to handle Redis usage with Spring Boot. Also some examples on how to error handle exceptions and issues with Redis.

The code below will help you initialize your redis connect and how to use it. One thing to take notice is that redis keys are global so you must make sure that any method parameter you use with you keys and unique. For this reason below you have samples of custom key generators.

Redis Samples

 

Redis main configurations


import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.cache.CacheManager;
import org.springframework.cache.annotation.*;
import org.springframework.cache.interceptor.CacheErrorHandler;
import org.springframework.cache.interceptor.KeyGenerator;
import org.springframework.context.annotation.*;
import org.springframework.context.support.PropertySourcesPlaceholderConfigurer;
import org.springframework.data.redis.cache.RedisCacheManager;
import org.springframework.data.redis.connection.jedis.JedisConnectionFactory;
import org.springframework.data.redis.core.RedisTemplate;

import org.springframework.data.redis.serializer.StringRedisSerializer;
import org.springframework.util.StringUtils;

import java.lang.reflect.Method;
import java.util.HashMap;
import java.util.Map;


@Configuration
@ComponentScan
@EnableCaching
@Profile({"dev","test"})
public class RedisCacheConfig extends CachingConfigurerSupport {
    @Override
    public CacheErrorHandler errorHandler() {

        return new CustomCacheErrorHandler();

    }

    protected final org.slf4j.Logger logger = LoggerFactory.getLogger(RedisCacheConfig.class);


    // This is a custom default keygenerator that is used if no other explicit key generator is specified
    @Bean
    public KeyGenerator keyGenerator() {
        return new KeyGenerator() {
            protected final org.slf4j.Logger logger = LoggerFactory.getLogger(RedisCacheConfig.class);

            @Override
            public Object generate(Object o, Method method, Object... objects) {
                return RedisCacheConfig.keyGeneratorProcessor(logger, o, method, null, objects);

            }
        };
    }

    // A custom key generator that generates a key based on the first method parameter while ignoring all other parameters
    @Bean("keyGeneratorFirstParamKey")
    public KeyGenerator keyGeneratorFirstParamKey() {

        return new KeyGenerator() {
            protected final org.slf4j.Logger logger = LoggerFactory.getLogger(RedisCacheConfig.class);

            @Override
            public Object generate(Object o, Method method, Object... objects) {

                return RedisCacheConfig.keyGeneratorProcessor(logger, o, method, 0, objects);
            }
        };
    }

    // A custom key generator that generates a key based on the second method parameter while ignoring all other parameters

    @Bean("keyGeneratorSecondParamKey")
    public KeyGenerator keyGeneratorSecondParamKey() {

        return new KeyGenerator() {
            protected final org.slf4j.Logger logger = LoggerFactory.getLogger(RedisCacheConfig.class);

            @Override
            public Object generate(Object o, Method method, Object... objects) {

                return RedisCacheConfig.keyGeneratorProcessor(logger, o, method, 1, objects);
            }
        };
    }

    // This is the main logic for creating cache keys
    public static String keyGeneratorProcessor(org.slf4j.Logger logger, Object o, Method method, Integer keyIndex, Object... objects) {

        // Retrieve all cache names for each anonation and compose a cache key prefix
        CachePut cachePutAnnotation = method.getAnnotation(CachePut.class);
        Cacheable cacheableAnnotation = method.getAnnotation(Cacheable.class);
        CacheEvict cacheEvictAnnotation = method.getAnnotation(CacheEvict.class);
        org.springframework.cache.annotation.CacheConfig cacheConfigClassAnnotation = o.getClass().getAnnotation(org.springframework.cache.annotation.CacheConfig.class);
        String keyPrefix = "";
        String[] cacheNames = null;

        if (cacheConfigClassAnnotation != null)
            cacheNames = cacheConfigClassAnnotation.cacheNames();


        if (cacheEvictAnnotation != null)
            if (cacheEvictAnnotation.value() != null)
                if (cacheEvictAnnotation.value().length > 0)
                    cacheNames = org.apache.commons.lang3.ArrayUtils.addAll(cacheNames, cacheEvictAnnotation.value());

        if (cachePutAnnotation != null)
            if (cachePutAnnotation.value() != null)
                if (cachePutAnnotation.value().length > 0)
                    cacheNames = org.apache.commons.lang3.ArrayUtils.addAll(cacheNames, cachePutAnnotation.value());

        if (cacheableAnnotation != null)
            if (cacheableAnnotation.value() != null)
                if (cacheableAnnotation.value().length > 0)
                    cacheNames = org.apache.commons.lang3.ArrayUtils.addAll(cacheNames, cacheableAnnotation.value());

        if (cacheNames != null)
            if (cacheNames.length > 0) {
                for (String cacheName : cacheNames)
                    keyPrefix += cacheName + "_";
            }

        StringBuilder sb = new StringBuilder();


        int parameterIndex = 0;
        for (Object obj : objects) {
            if (obj != null && !StringUtils.isEmpty(obj.toString())) {
                if (keyIndex == null)
                    sb.append(obj.toString());
                else if (parameterIndex == keyIndex) {
                    sb.append(obj.toString());
                    break;
                }
            }
            parameterIndex++;
        }


        String fullKey = keyPrefix + sb.toString();

        logger.debug("REDIS KEYGEN for CacheNames: " + keyPrefix + " with KEY: " + fullKey);

        return fullKey;
        //---------------------------------------------------------------------------------------------------------

        // Another example how to do custom cache keys
        // This will generate a unique key of the class name, the method name,
        // and all method parameters appended.
                /*StringBuilder sb = new StringBuilder();
                sb.append(o.getClass().getName());
                sb.append("-" + method.getName() );
                for (Object obj : objects) {
                    if(obj != null)
                        sb.append("-" + obj.toString());
                }

                if(logger.isDebugEnabled())
                    logger.debug("REDIS KEYGEN: " + sb.toString());
                return sb.toString();*/
    }

    // Create the redis connection here
    @Bean
    public JedisConnectionFactory jedisConnectionFactory() {
        JedisConnectionFactory jedisConFactory = new JedisConnectionFactory();

        jedisConFactory.setUseSsl(true);
        jedisConFactory.setHostName("127.0.0.1");
        jedisConFactory.setPort(6379);

        if (!StringUtils.isEmpty(mytoken)) {
            jedisConFactory.setPassword(mytoken);
        }

        jedisConFactory.setUsePool(true);
        jedisConFactory.afterPropertiesSet();

        return jedisConFactory;
    }

    @Bean
    public static PropertySourcesPlaceholderConfigurer propertySourcesPlaceholderConfigurer() {
        return new PropertySourcesPlaceholderConfigurer();
    }

    @Bean
    public RedisTemplate redisTemplate() {
        RedisTemplate redisTemplate = new RedisTemplate();
        redisTemplate.setConnectionFactory(jedisConnectionFactory());
        redisTemplate.setKeySerializer(new StringRedisSerializer());

        return redisTemplate;
    }

    // Cache configurations like how long data is cached
    @Bean
    public CacheManager cacheManager(RedisTemplate redisTemplate) {
        RedisCacheManager cacheManager = new RedisCacheManager(redisTemplate);

        Map cacheExpiration = new HashMap();


        cacheExpiration.put("USERS", 120);
        cacheExpiration.put("CARS", 3600):

        // Number of seconds before expiration. Defaults to unlimited (0)
        cacheManager.setDefaultExpiration(60);
        cacheManager.setExpires(cacheExpiration);
        return cacheManager;
    }
}

 

Redis Error/Exception Handling

 

public class CustomCacheErrorHandler implements CacheErrorHandler {


    protected final org.slf4j.Logger logger = LoggerFactory.getLogger(this.getClass());

    protected Gson gson = new GsonBuilder().create();


    @Override

    public void handleCacheGetError(RuntimeException exception,

                                    Cache cache, Object key) {

        logger.error("Error in REDIS GET operation for KEY: " + key, exception);
        try
        {
            if (cache.get(key) != null && logger.isDebugEnabled())
                logger.debug("Possible existing data which for the cache GET operation in REDIS Cache by KEY: " + key + " with TYPE: " + cache.get(key).get().getClass() + " and DATA: " + this.gson.toJson(cache.get(key).get()));
        } catch (Exception ex)
        {
            // NOTICE: This exception is not logged because this might occur because the cache connection is not established.
            // So if the initial exception that was thrown might have been the same, no connection to the cache server.
            // In such a case this is logged in above already, before the try catch.
        }
    }

    @Override

    public void handleCachePutError(RuntimeException exception, Cache cache,

                                    Object key, Object value) {

        logger.error("Error in REDIS PUT operation for KEY: " + key, exception);
        if(logger.isDebugEnabled())
            logger.debug("Error in REDIS PUT operation for KEY: " + key + " with TYPE: " + value.getClass() + " and DATA: " + this.gson.toJson(value), exception);
    }

    @Override

    public void handleCacheEvictError(RuntimeException exception, Cache cache,

                                      Object key) {

        logger.error("Error in REDIS EVICT operation for KEY: " + key, exception);
        try
        {
            if (cache.get(key) != null  && logger.isDebugEnabled())
                logger.debug("Possible existing data which for the cache EVICT operation in REDIS Cache by KEY: " + key + " with TYPE: " + cache.get(key).get().getClass() + " and DATA: " + this.gson.toJson(cache.get(key).get()));
        } catch (Exception ex)
        {
            // NOTICE: This exception is not logged because this might occur because the cache connection is not established.
            // So if the initial exception that was thrown might have been the same, no connection to the cache server.
            // In such a case this is logged in above already, before the try catch.
        }
    }

    @Override

    public void handleCacheClearError(RuntimeException exception,Cache cache){
        logger.error("Error in REDIS CLEAR operation ", exception);
    }

}

Custom Key Generator Example

 
@Cacheable(value = "USERS", keyGenerator = "keyGeneratorFirstParamKey")
    public UserData getUsers(String userId, Object data)
    {
        // Do something here
    }

Advertisements

Java + Spring Boot: Explicit Class Instances with Profiles, Beans and Qualifiers

Here is a simple example how to use beans to create instances of classes based on their profiles.

Firstly, you need is a base class or an interface that each class will inherit/implement.

The simple way of doing this is to just simply using the @Profile annotation on a class with the desired profile name. For example:

Use a dev profile for a class that is created when you dev profile is up and running and test profile when your test profile is used.

Then simply use the @Autowired annotation to on the base class/interface. The rest if induced automatically based on your profile. BUT this approach works fine in your classes and/or interface reside within the same package.

In a case that you are using a base class or an interface from another library/package and wanting to create a different class to be used with different profiles this might not work because you can’t change the used profile name in the base library/package.

In these cases you do the following:

  1. Create @Bean functions that return instantiate the desired class into an object based on a profile set to the bean function. The return value can be the base class or interface.
  2. Give the same bean name to all functions.
  3. On the @Autowired class member add the @Qualifier annotation giving the bean name which you want.

Spring will in this case induce the right object instance based on the profile in defined on a bean function.

@Bean(name="authenticationLogic")
    @Profile("dev")
    public BaseAuthentication getBaseAuth()
    {
        return new MockAuthenticationClient();
    }

    @Bean(name="authenticationLogic")
    @Autowired
    @Profile("test")
    public BaseAuthentication getMainAuth(MessageService messageService)
    {
        return new MainAuthClient(messageService);
    }



    @Autowired
    @Qualifier("authenticationLogic")
    private BaseAuthentication baseAuthentication;

Adding git information to your Spring Actuator Info endpoint with Gradle

Hi,

This is how you can add git related information in case you need that to keep track what functionality and code your development or test environments are using.

Configuration

First you need to add the following to your Gradle file:

plugins {
   id "com.gorylenko.gradle-git-properties" version "1.4.21"
}

apply plugin: 'com.gorylenko.gradle-git-properties'

After this you should have a new Gradle task that will generate a git.properties file that your Actuator Info Endpoint can use. This file by default is generated into the build path resources folder. So run this command before building your jar or docker image etc.

gradle generateGitProperties

Bonus

If you want to access the Info actutor enpoint to display that info from somewhere else you can do this:
@Autowired
InfoEndpoint infoEndpoint;

return new JSONObject(this.infoEndpoint.invoke()).toString();

Links

https://docs.spring.io/spring-boot/docs/current/reference/htmlsingle/#howto-git-info

https://github.com/n0mer/gradle-git-properties

Helper Scripts for Docker, git and Java developers

Hi,

Here are some of my own scripts that I use when developing to ease my life:

Building a Java Gradle project, then building a docker image and pushing it

./gradlew test
if [ $? -eq 0 ]; then
    echo Tests OK
    gradle clean
    gradle generateGitProperties
    gradle bootRepackage
    ./cleandocker.sh
    docker rmi {your image name + tag}
    docker build -t {your image name + tag} .
    ./dockerregistrylogin.sh
    docker push {your image name + tag}
else
    echo Tests Failed
    exit 1
fi

Clean docker from all running containers and stopped ones

echo "Stoping all containers"
docker stop $(docker ps -a -q)
echo "Removing all containers"
docker rm $(docker ps -a -q)
echo "Starting dev environment"

Commit your code to git after gradle tests are successfull

./gradlew test
if [ $? -eq 0 ]; then
    echo Tests OK
    git add .
    git commit -m "$1"
    git push
else
    echo Tests Failed
    exit 1
fi

Merge your branch with your master

git checkout master
git pull origin master
git merge dev -m "$1"
git push origin master
git checkout dev

This one is for AWS Developers to run and get the AWS ECR docker login

#Notice: To use a certain profile for login define additional profiles like this: aws configure --profile awscli

function doAwsDockerRegistryLogin()
{
    local  myresult=$(aws ecr get-login --no-include-email --region eu-central-1 --profile awscli)
    echo "$myresult"
}

result=$(doAwsDockerRegistryLogin)   # or result=`myfunc`
eval $result

 

AWS ECS and Bitbucket Pipelines: Deploy your docker application

Hi,

Here are some tips and tricks on how to update an existing AWS ECS deployment.

NOTICE: This post assume that you have some knowledge on AWS, scripting, docker and Bitbucket.

The scripts and guide below will do the following:

  1. Clone external libraries needed for your project (Assumes that you have a multi-project application
  2. Build your application
  3. Build the docker image
  4. Push the docker image into Elastic Container Service own container registry
  5. Stop all tasks in a cluster to force the associated services to restart the tasks
    1. I am using this method of deploying to avoid making constant new task definitions. I find that unnecessary. My view is to have a deployment docker tag that your target service task definitions use. In this way you only make sure that you have one good task definition that you plan to you. If needed later update your task definition and service to use it. This deployment suggestion will not care of any other detail that except the docker image and cluster name.
  6. Then test your API or Web App in the cloud with some 3rd party tool in this case I am using Postman collection, Postman Environmental Settings and Newman

The above steps can be performed automatically when you make changes to a branch or manually from the commit view or branches view (on the row with the branch or commit id move you mouse on top of “…” to get the option “Run pipeline for a branch” and selected the manual pipeline option)

Needed steps:

  1. Create/have an IAM access keys for deployment into ECS and ECR from Bitbucket.
  2. Generate SSH keys in the Bitbucket repository where you plan to run your pipeline
  3. If you have any depended Bitbucket repositories copy the Public Key in Step 2 into that repository settings.
  4. Then in the primary repository from which you plan to deploy set environmental variables needed for the deployment.
  5. Create you pipeline with the example Bitbucket pipeline definition file and supplement scripts.

Step 1: AWS Access

You will need an access key/secrect to AWS with the following minimum policy settings:


{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor0",
            "Effect": "Allow",
            "Action": [
                "ecs:ListTasks",
                "ecr:CompleteLayerUpload",
                "ecr:GetAuthorizationToken",
                "ecs:StopTask",
                "ecr:UploadLayerPart",
                "ecr:InitiateLayerUpload",
                "ecr:BatchCheckLayerAvailability",
                "ecr:PutImage"
            ],
            "Resource": "*"
        }
    ]
}

Step 2: SSH Keys for Bitbucket

More info here on how to generate a key:

https://confluence.atlassian.com/bitbucket/use-ssh-keys-in-bitbucket-pipelines-847452940.html?_ga=2.23419699.803528104.1518767114-2088289616.1517306997

Notice: Remember to get the public key

Step 3: Other repositories (Optional)

From bitbucket:

If you want your Pipelines builds to be able to access a different Bitbucket repository (other than the repo where the builds run):

  1. Add an SSH key to the settings for the repo where the build will run, as described in Step 1 above (you can create a new key in Bitbucket Pipelines or use an existing key).
  2. Add the public key from that SSH key pair directly to settings for the other Bitbucket repo (i.e. the repo that your builds need to have access to).
    See Use access keys for details on how to add a public key to a Bitbucket repo.

Step 4: Setting up environmental variables

APPIMAGE_TESTENV_CLUSTER : The cluster name where to which the docker image is deployed to  in this case a test environment that is manually triggered

APPIMAGE_DEVENV_CLUSTER: A dev target cluster that is associated with the master branch and starts automatically

APPIMAGE_NAME: The docker image name (Notice: Must match the one in your service -> task definition)

APPIMAGE_TAG: the docker image tag The docker image name (Notice: Must match the one in your service -> task definition)

AWS_ACCESS_KEY_ID (SECURE)

AWS_SECRET_ACCESS_KEY (SECURE)

AWS_DEFAULT_REGION : The region where your cluster is located

REGISTRYNAME : The ECR registry name wherethe image is to be pushed

Step 5: Bitbucket Pipeline

Pipeline definitions

The sample pipeline script has two options:
* Custom/manual deployment in the custom section of the script
* Branches/automatic deployment in the branches section of the script

# This is a sample build configuration for Java (Gradle).
# Check our guides at https://confluence.atlassian.com/x/zd-5Mw for more examples.
# Only use spaces to indent your .yml configuration.
# -----
# You can specify a custom docker image from Docker Hub as your build environment.
image: atlassian/default-image:latest # Include Java support
options:
  max-time: 15 # 15 minutes incase something hangs up
  docker: true # Include Docker support
pipelines:
  custom: # Pipelines that can only be triggered manually
    test-env:
      - step:
          caches:
            - gradle
          script:
            # Modify the commands below to build your repository.
            # You must commit the Gradle wrapper to your repository
            # https://docs.gradle.org/current/userguide/gradle_wrapper.html
            - git clone {your external repository}
            - ls
            - bash ./scripts/bitbucket/buildproject.sh
            # Install AWS CLI and configure it
            #----------------------------------------
            - apt-get update
            - apt-get install jq
            - curl "https://s3.amazonaws.com/aws-cli/awscli-bundle.zip" -o "awscli-bundle.zip"
            - unzip awscli-bundle.zip
            - ./awscli-bundle/install -b ~/bin/aws
            - export PATH=~/bin:$PATH
            #----------------------------------------
            - bash ./scripts/bitbucket/awsdev-dockerregistrylogin.sh
            - export IMAGE_NAME=$REGISTRYNAME/$APPIMAGE_NAME:$APPIMAGE_TAG
            - docker build -t $IMAGE_NAME .
            - docker push $IMAGE_NAME
            # This will stop all tasks in the AWS Cluster, by doing this the AWS Service will start the defined task definitions as new tasks.
            # NOTICE: This approach needs task definitions attached to services and no manually started tasks.
            - bash ./scripts/bitbucket/stopalltasks.sh $APPIMAGE_TESTENV_CLUSTER
            #----------------------------------------
            # Install Newman tool and test with postman collection and environmental settings your web app
            - npm install -g newman
            - ./scripts/newman-API-tests/run-testenv-tests.sh
            #----------------------------------------
    prod-env:
      - step:
          script:
            - echo "Manual triggers for deployments are awesome!"
  branches:
    master:
      - step:
          caches:
            - gradle
          script:
            #----------------------------------------
            # Modify the commands below to build your repository.
            # You must commit the Gradle wrapper to your repository
            # https://docs.gradle.org/current/userguide/gradle_wrapper.html
            - git clone {your external repository}
            - ls
            - bash ./scripts/bitbucket/buildproject.sh
            # Install AWS CLI and configure it
            #----------------------------------------
            - apt-get update
            - apt-get install jq
            - curl "https://s3.amazonaws.com/aws-cli/awscli-bundle.zip" -o "awscli-bundle.zip"
            - unzip awscli-bundle.zip
            - ./awscli-bundle/install -b ~/bin/aws
            - export PATH=~/bin:$PATH
            #----------------------------------------
            # Build and install the newest docker image
            - bash ./scripts/bitbucket/awsdev-dockerregistrylogin.sh
            - export IMAGE_NAME=$REGISTRYNAME/$APPIMAGE_NAME:$APPIMAGE_TAG
            - docker build -t $IMAGE_NAME .
            - docker push $IMAGE_NAME
            #----------------------------------------
            # This will stop all tasks in the AWS Cluster, by doing this the AWS Service will start the defined task definitions as new tasks.
            # NOTICE: This approach needs task definitions attached to services and no manually started tasks.
            - bash ./scripts/bitbucket/stopalltasks.sh $APPIMAGE_DEVENV_CLUSTER
            #----------------------------------------
            # Install Newman tool and test with postman collection and environmental settings your web app
            - npm install -g newman
            - ./scripts/newman-API-tests/run-devenv-tests.sh
            #----------------------------------------

Stop all AWS tasks in the cloud

#!/usr/bin/env bash

echo "Getting tasks from AWS:"
echo "Cluster: $1 Service: $2"
#For a single task
#----------------
#task=$(aws ecs list-tasks --cluster "$1" --service-name "$2" | jq --raw-output '.taskArns[0] | split("/")[1]' )
 #echo "Stopping task: " $task

 #aws ecs stop-task --task $task --cluster "$1"
#----------------
tasks=$(aws ecs list-tasks --cluster "$1" --service-name "$2" | jq -r '.taskArns | map(.[0:]) | reduce .[] as $item (""; . + $item + " ")')
echo "Tasks received"
for task in $tasks; do
echo "Stopping task from AWS: " $task
	aws ecs stop-task --task $task --cluster "$1"
#echo "Task stopped."
done

Build your project

echo "Rebuilding project"
#gradlew_output=$(./gradlew build);
#echo "$gradlew_output"

./gradlew test
if [ $? -eq 0 ]; then
    echo Tests OK
    ./gradlew clean
    ./gradlew bootRepackage
else
    echo Tests Failed
fi

Get the AWS Login details for ECR docker login

#Notice: To use a certain profile for login define additional profiles like this: aws configure --profile awscli

function doAwsDockerRegistryLogin()
{
    local  myresult=$(aws ecr get-login --no-include-email)
    echo "$myresult"
}

result=$(doAwsDockerRegistryLogin)   # or result=`myfunc`
eval $result

Running API or WebApp tests with Newman and Postman

What do you need to make the tests work

  1. Create a new Postman collection
  2. Add your URLs to test
  3. Add scripts into the test tab
  4. When all your URLs in your collection are ready export them from the collection … button
  5. (Optional) Then create environment settings that you can export and use with newman

Bash script to run the newman tests

sleep 1m # Force a wait to make sure that all AWS services, your app, LBs etc are all loaded up and running

echo $DEVENV_URL
until $(curl --output /dev/null --silent --head --fail --insecure "$DEVENV_URL"); do
    printf '.'
    sleep 5
done

echo "Starting newman tests"
newman run {your postman collection}.postman_collection.json --environment "{your postman collection}.postman_environment.json" --insecure --delay-request 10<span id="mce_SELREST_start" style="overflow:hidden;line-height:0;"></span>

Postman scripts example

Retrieving a token from the body and inserting it into an environmental variable

var jsonData = JSON.parse(responseBody);

console.log("TOKEN:" + jsonData.token);

var str_array = jsonData.token.split('.');
for(var i = 0; i < str_array.length -1; i++) {
console.log("Array Item: " + i);
console.log(str_array[i])
console.log(CryptoJS.enc.Utf8.stringify(CryptoJS.enc.Base64.parse(str_array[i])));
}
postman.setEnvironmentVariable("token", jsonData.token);

Testing a response for success and body content

// example using pm.response.to.be*
pm.test("response must be valid and have a body", function () {
// assert that the status code is 200
pm.response.to.be.ok; // info, success, redirection, clientError, serverError, are other variants
// assert that the response has a valid JSON body
pm.response.to.be.withBody;
pm.response.to.be.json; // this assertion also checks if a body exists, so the above check is not needed
});

console.log("BODY:" + responseBody);

Links

http://2mins4code.com/2017/11/08/building-a-cicd-environment-with-bitbucket-pipelines-docker-aws-ecs-for-an-angular-app/

https://bitbucket.org/awslabs/amazon-ecs-bitbucket-pipelines-python/overview

https://confluence.atlassian.com/bitbucket/deploy-to-amazon-ecs-892623902.html

https://bitbucket-pipelines.prod.public.atl-paas.net/validator

https://confluence.atlassian.com/bitbucket/configure-bitbucket-pipelines-yml-792298910.html

https://confluence.atlassian.com/bitbucketserver/getting-started-with-bitbucket-server-and-aws-776640193.html

https://confluence.atlassian.com/bitbucket/java-with-bitbucket-pipelines-872013773.html

https://confluence.atlassian.com/bitbucket/use-ssh-keys-in-bitbucket-pipelines-847452940.html?_ga=2.23419699.803528104.1518767114-2088289616.1517306997

https://confluence.atlassian.com/bitbucket/environment-variables-794502608.html

https://docs.aws.amazon.com/cli/latest/reference/index.html#cli-aws

https://confluence.atlassian.com/bitbucket/run-pipelines-manually-861242583.html

https://www.npmjs.com/package/newman

http://blog.getpostman.com/2017/10/25/writing-tests-in-postman/

How to use: Azure Service Bus – Notification hub, Azure hosted SignalR Hub with .NET clients, Android clients and web JavaScript clients

Huh that is a looong title for a blog post :). It is easier for me to write these all in one post since they are related the code in this case. So here I go, bear with me, it’s going to be alot of stuff and code.

The codes and examples here are from the own personal technology workbench project hosted in Azure. More code and examples how these features and functionality work can be found here.

I will go in the following order with the topics covered in this post:

  • Notification Hub
    • Setting up the Azure Service Bus – Notification Hub
    • Connecting with a .NET Client and sending messages
    • Connecting and listening to messages with an Android device client
  • SignalR Hub
    • Setting up a Azure hosted SignalR hub
    • Connecting and listening to activities with the following clients:
      • Connecting with the .NET Client
      • Connecting with the JavaScript client
      • Connecting with the Android client

Notification Hub

Some links to get you started:

http://azure.microsoft.com/en-us/documentation/articles/mobile-services-dotnet-backend-android-get-started-push/

http://azure.microsoft.com/en-us/documentation/articles/mobile-services-dotnet-backend-android-push-notifications-app-users/

Setting up the Azure Service Bus – Notification Hub

I wont go into much details here. Microsoft has done a great job documenting these steps, I recommend you go to the following link and follow the first part of these instructions:

http://azure.microsoft.com/en-us/documentation/articles/mobile-services-dotnet-backend-android-get-started-push/

What you will need for my code example is a Google Developer account and a API Key from google to be able to send and receive GCM messages to Android devices. Of course you will also need a Azure account to be able to create the notification hub.

Add the following Nugget package:

Microsoft Azure Service Bus

Connecting with a .NET Client and sending messages

The code below will connect from a .NET Client to the Notification Hub. This is done in the class constructor. At the moment this code can only send GCM messages to Android devices. Note that these operations are asynchronous.

https://github.com/lionadi/MyFitnessTracker/blob/master/MyFitnessTrackerLibrary/ServiceBus/NotificationGateway.cs

But you could use a "central" hub from where to send these messages which will handle sending messages to different devices through a WebAPI:
https://github.com/lionadi/MyFitnessTracker/tree/master/FitTrackerHubCentral
https://github.com/lionadi/MyFitnessTracker/blob/master/FitTrackerHubCentral/FitTrackerHubCentral/Controllers/NotificationsController.cs



using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using Microsoft.ServiceBus.Notifications;

namespace MyFitnessTrackerLibrary.ServiceBus
{
    // TODO: Replace this with a connection to the notification central hub, do not sent message directly from here in the future!!!!
    public class NotificationGateway
    {
        private NotificationHubClient hub = null;
        private static NotificationGateway _notificationGateway;
        public NotificationGateway()
        {

            hub = NotificationHubClient.CreateClientFromConnectionString(MyFitnessTrackerLibrary.Globals.MyFitAppSettings.NotificationHubConnectionString, MyFitnessTrackerLibrary.Globals.MyFitAppSettings.NotificationHubName);

        }

        ~NotificationGateway()
        {

        }

        public async Task<NotificationOutcome> SendMessage(string message)
        {
            var toast = "{ \"data\" : {\"message\":\"" + "From : " + message + "\"}}";
            return await hub.SendGcmNativeNotificationAsync(toast);
        }

        public static NotificationGateway GetInstance()
        {
            if(_notificationGateway == null)
            {
                _notificationGateway = new NotificationGateway();
            }

            return _notificationGateway;
        }
    }
}

Connecting and listening to messages with an Android device client

The Android side is a bit more complicated and annoying. You have to do more work here.

At the first you need two Microsoft java libraries to be able to connect to a notification hub:

https://github.com/lionadi/MyFitnessTracker/blob/master/MyFitnessAndroidApp/app/libs/notification-hubs-0.4.jar

https://github.com/lionadi/MyFitnessTracker/blob/master/MyFitnessAndroidApp/app/libs/notifications-1.0.1.jar

After this you need to add them to your Android Studio gradle file:

https://github.com/lionadi/MyFitnessTracker/blob/master/MyFitnessAndroidApp/app/build.gradle

dependencies {
 compile fileTree(dir: 'libs', include: ['*.jar'])
 compile 'com.android.support:appcompat-v7:21.0.3'
 compile 'com.google.android.gms:play-services:6.5.87'
 compile 'com.google.code.gson:gson:2.3.1'
 compile files('libs/signalr-client-sdk.jar')
compile files('libs/notifications-1.0.1.jar')
compile files('libs/notification-hubs-0.4.jar')
 compile files('libs/Java-WebSocket-1.3.0.jar')
 compile project(':signalr-client-sdk-android-release')
}

 Please remember to follow these instructions to setup your Android Studio project in a correct manner, more details here:

http://azure.microsoft.com/en-us/documentation/articles/mobile-services-dotnet-backend-android-get-started-push/

http://azure.microsoft.com/fi-fi/documentation/articles/notification-hubs-aspnet-backend-android-notify-users/

The most important piece of code is the class named MyHandler in this case which will handle your notifications once your device is registered to the notification hub:

https://github.com/lionadi/MyFitnessTracker/blob/master/MyFitnessAndroidApp/app/src/main/java/com/example/adriansimionescu/myfitnessandroidapp/MyHandler.java


package com.example.adriansimionescu.myfitnessandroidapp;

import android.app.NotificationManager;
import android.app.PendingIntent;
import android.content.Context;
import android.content.Intent;
import android.os.Bundle;
import android.support.v4.app.NotificationCompat;
import com.microsoft.windowsazure.notifications.NotificationsHandler;

public class MyHandler extends NotificationsHandler {
    public static final int NOTIFICATION_ID = 1;
    private NotificationManager mNotificationManager;
    NotificationCompat.Builder builder;
    Context ctx;

    static public MainActivity mainActivity;

    @Override
    public void onReceive(Context context, Bundle bundle) {
        ctx = context;
        String nhMessage = bundle.getString("message");

        sendNotification(nhMessage);
        mainActivity.DialogNotify("Received Notification",nhMessage);
    }

    private void sendNotification(String msg) {
        mNotificationManager = (NotificationManager)
                ctx.getSystemService(Context.NOTIFICATION_SERVICE);

        PendingIntent contentIntent = PendingIntent.getActivity(ctx, 0,
                new Intent(ctx, MainActivity.class), 0);

        NotificationCompat.Builder mBuilder =
                new NotificationCompat.Builder(ctx)
                        .setContentTitle("Notification Hub Demo")
                        .setStyle(new NotificationCompat.BigTextStyle()
                                .bigText(msg))
                        .setContentText(msg);

        mBuilder.setContentIntent(contentIntent);
        mNotificationManager.notify(NOTIFICATION_ID, mBuilder.build());
    }
}</pre>
<pre>

You also need a class that will register you device to the notification hub:
https://github.com/lionadi/MyFitnessTracker/blob/master/MyFitnessAndroidApp/app/src/main/java/com/example/adriansimionescu/myfitnessandroidapp/RegisterClient.java

 


package com.example.adriansimionescu.myfitnessandroidapp;

import java.io.IOException;
import java.io.UnsupportedEncodingException;
import java.util.Set;

import org.apache.http.HttpResponse;
import org.apache.http.HttpStatus;
import org.apache.http.client.ClientProtocolException;
import org.apache.http.client.HttpClient;
import org.apache.http.client.methods.HttpPost;
import org.apache.http.client.methods.HttpPut;
import org.apache.http.client.methods.HttpUriRequest;
import org.apache.http.entity.StringEntity;
import org.apache.http.impl.client.DefaultHttpClient;
import org.apache.http.util.EntityUtils;
import org.json.JSONArray;
import org.json.JSONException;
import org.json.JSONObject;

import android.content.Context;
import android.content.SharedPreferences;
import android.util.Log;

public class RegisterClient {
    private static final String PREFS_NAME = "ANHSettings";
    private static final String REGID_SETTING_NAME = "ANHRegistrationId";
    private String Backend_Endpoint;
    SharedPreferences settings;
    protected HttpClient httpClient;
    private String authorizationHeader;

    public RegisterClient(Context context, String backendEnpoint) {
        super();
        this.settings = context.getSharedPreferences(PREFS_NAME, 0);
        httpClient =  new DefaultHttpClient();
        Backend_Endpoint = backendEnpoint + "/api/register";
    }

    public String getAuthorizationHeader() {
        return authorizationHeader;
    }

    public void setAuthorizationHeader(String authorizationHeader) {
        this.authorizationHeader = authorizationHeader;
    }

    public void register(String handle, Set<String> tags) throws ClientProtocolException, IOException, JSONException {
        String registrationId = retrieveRegistrationIdOrRequestNewOne(handle);

        JSONObject deviceInfo = new JSONObject();
        deviceInfo.put("Platform", "gcm");
        deviceInfo.put("Handle", handle);
        deviceInfo.put("Tags", new JSONArray(tags));

        int statusCode = upsertRegistration(registrationId, deviceInfo);

        if (statusCode == HttpStatus.SC_OK) {
            return;
        } else if (statusCode == HttpStatus.SC_GONE){
            settings.edit().remove(REGID_SETTING_NAME).commit();
            registrationId = retrieveRegistrationIdOrRequestNewOne(handle);
            statusCode = upsertRegistration(registrationId, deviceInfo);
            if (statusCode != HttpStatus.SC_OK) {
                Log.e("RegisterClient", "Error upserting registration: " + statusCode);
                throw new RuntimeException("Error upserting registration");
            }
        } else {
            Log.e("RegisterClient", "Error upserting registration: " + statusCode);
            throw new RuntimeException("Error upserting registration");
        }
    }

    private int upsertRegistration(String registrationId, JSONObject deviceInfo)
            throws UnsupportedEncodingException, IOException,
            ClientProtocolException {
        HttpPut request = new HttpPut(Backend_Endpoint+"/"+registrationId);
        request.setEntity(new StringEntity(deviceInfo.toString()));
        request.addHeader("Authorization", "Basic "+authorizationHeader);
        request.addHeader("Content-Type", "application/json");
        HttpResponse response = httpClient.execute(request);
        int statusCode = response.getStatusLine().getStatusCode();
        return statusCode;
    }

    private String retrieveRegistrationIdOrRequestNewOne(String handle) throws ClientProtocolException, IOException {
        if (settings.contains(REGID_SETTING_NAME))
            return settings.getString(REGID_SETTING_NAME, null);

        HttpUriRequest request = new HttpPost(Backend_Endpoint+"?handle="+handle);
        request.addHeader("Authorization", "Basic "+authorizationHeader);
        HttpResponse response = httpClient.execute(request);
        if (response.getStatusLine().getStatusCode() != HttpStatus.SC_OK) {
            Log.e("RegisterClient", "Error creating registrationId: " + response.getStatusLine().getStatusCode());
            throw new RuntimeException("Error creating Notification Hubs registrationId");
        }
        String registrationId = EntityUtils.toString(response.getEntity());
        registrationId = registrationId.substring(1, registrationId.length()-1);

        settings.edit().putString(REGID_SETTING_NAME, registrationId).commit();

        return registrationId;
    }
}

After all these steps and setups you can finally go to your activity and add the following pieces of codes to fire up the connection and start listening to messages:
https://github.com/lionadi/MyFitnessTracker/blob/master/MyFitnessAndroidApp/app/src/main/java/com/example/adriansimionescu/myfitnessandroidapp/MainActivity.java

import com.microsoft.windowsazure.messaging.*;
import com.microsoft.windowsazure.notifications.NotificationsManager;

// Define this properties in you activity class</pre>
<pre>private RegisterClient registerClient;
private String SENDER_ID = "";
private GoogleCloudMessaging gcm;
private NotificationHub hub;
private String HubName = "fittracker";
private String HubListenConnectionString = "";</pre>
<pre>
@SuppressWarnings("unchecked")
private void registerWithNotificationHubs() {
 new AsyncTask() {
 @Override
 protected Object doInBackground(Object... params) {
 try {
 String regid = gcm.register(SENDER_ID);
 DialogNotify("Registered Successfully", "RegId : " +
 hub.register(regid).getRegistrationId());
 } catch (Exception e) {
 DialogNotify("Exception",e.getMessage());
 return e;
 }
 return null;
 }
 }.execute(null, null, null);
}

/**
 * A modal AlertDialog for displaying a message on the UI thread
 * when theres an exception or message to report.
 *
 * @param title Title for the AlertDialog box.
 * @param message The message displayed for the AlertDialog box.
 */
public void DialogNotify(final String title,final String message)
{
 final AlertDialog.Builder dlg;
 dlg = new AlertDialog.Builder(this);

 runOnUiThread(new Runnable() {
 @Override
 public void run() {
 AlertDialog dlgAlert = dlg.create();
 dlgAlert.setTitle(title);
 dlgAlert.setButton(DialogInterface.BUTTON_POSITIVE,
 (CharSequence) "OK",
 new DialogInterface.OnClickListener() {
 public void onClick(DialogInterface dialog, int which) {
 dialog.dismiss();
 }
 });
 dlgAlert.setMessage(message);
 dlgAlert.setCancelable(false);
 dlgAlert.show();
 }
 });
}</pre>
<pre>

So the first function will register you device and the next one will create a notification with a message in your device. Next you create the connection with this piece of code:


<pre>MyHandler.mainActivity = this;
NotificationsManager.handleNotifications(this, SENDER_ID, MyHandler.class);
gcm = GoogleCloudMessaging.getInstance(this);
hub = new NotificationHub(HubName, HubListenConnectionString, this);
registerWithNotificationHubs();</pre>

Notice how you pass your activity instance you the myhandler class. This is important.

And that’s it :D. Simple yet alot of work. Luckily Microsoft has made a good job documenting these steps. If in trouble don’t hesitate to look up on some documentation.

SignalR Hub

Implementing SignalR on the .NET side is rather easy BUT there is so much automation that it simply feel weird :). Everything seems to work if you just follow the instuctions but as I noticed one you poke around custom authentication and authorization you can really mess thing up. For example I wanted to add to my SignalR hun web project EDM mapping to a database or a custom authentication, well I made the mistake of choosing to use Entity Framework version 6.0 which uses a different version of Newtonsoft.json library which caused all sorts of problems. Another issue which I ran into was that I create a connection identification by client ID to be able to notify the client devices and services of changes within my system. At one point I did a simple mistake of forgeting to pass on the identification information which lead to weird errors on the client side browser such as IE and Chrome. The errors had nothing to do with the fact that the connection failed on the server side because the user ID was missing. The browsers expressed errors related to CORS which made no sense since I configured CORS support. So just be careful.Setting up a

Azure hosted SignalR hub

Start by looking at this source:

http://www.asp.net/signalr/overview/getting-started/tutorial-getting-started-with-signalr

To host my SignalR hub in Azure I simply created an empty web application and followed the instructions in the link above. Sample code:

https://github.com/lionadi/MyFitnessTracker/blob/master/SignalRGateway/SignalRGateway/ChatHub.cs

https://github.com/lionadi/MyFitnessTracker/blob/master/SignalRGateway/SignalRGateway/index.html

Also include the following SignalR nuget package:

Microsoft ASP .NET SignalR (to be able to host)

Microsoft ASP .NET Cross-Origin Support

Windows Azure Storage

Lets look at bit more closely at the ChartHub:


using System;
using System.Collections.Generic;
using System.Linq;
using System.Web;
using System.Threading.Tasks;
using Microsoft.AspNet.SignalR;
using SignalRGateway.AzureTableStorage;
using System.Configuration;

using Microsoft.WindowsAzure;
using Microsoft.WindowsAzure.Storage;
using Microsoft.WindowsAzure.Storage.Table;
using MyFitnessTrackerLibrary.Globals;

namespace SignalRGateway
{
 public class ChatHub : Hub
 {

 public void Send(string name, string message)
 {
 // Call the broadcastMessage method to update clients.
 var queryResult = this.SendMessageTo(name, message);
 foreach (var entity in queryResult)
 {
 Clients.Client(entity.RowKey).broadcastMessage(name, message);
 }
 }

 public void IsDataUpdateRequiredForWeb(string name, bool isRequired, string message)
 {
 var queryResult = this.SendMessageTo(name, message);
 foreach (var entity in queryResult)
 {
 Clients.Client(entity.RowKey).isDataUpdateRequiredForWeb(name, isRequired, message);
 }
 Clients.All.isDataUpdateRequiredForWeb(name, isRequired, message);
 }

 public void IsDataUpdateRequiredForMobileClient(string name, bool isRequired, string message)
 {
 var queryResult = this.SendMessageTo(name, message);
 foreach (var entity in queryResult)
 {
 Clients.Client(entity.RowKey).isDataUpdateRequiredForMobileClient(name, isRequired, message);
 }
 }

 private List<ConnectionEntity> SendMessageTo(String who, String message)
 {
 //var name = Context.User.Identity.Name;
 var name = this.GetConnectionUser();

 if (!String.IsNullOrEmpty(name))
 {
 var table = GetConnectionTable();

 // Notice that the partition keys are stored in azure storage as lower case
 var query = new TableQuery<ConnectionEntity>()
 .Where(TableQuery.GenerateFilterCondition(
 "PartitionKey",
 QueryComparisons.Equal,
 who.ToLowerInvariant()));

 var queryResult = table.ExecuteQuery(query).ToList();
 if (queryResult.Count == 0)
 {
 Clients.Caller.showErrorMessage("The user is no longer connected.");
 }
 else
 {
 // Load only once the host application connections to display the data there
 if(queryResult.Count(o=>o.PartitionKey.Equals(Constants.SignalR_HostApplicationUserName.ToLowerInvariant())) <= 0)
 queryResult.AddRange(this.SendMessageTo(Constants.SignalR_HostApplicationUserName, message));

 return queryResult;
 }
 }

 return new List<ConnectionEntity>();
 }

 // This assumes that "normmaly" all others clients than the host SignalR web application (this app) will use header named as username for user identification. The SignalR web app will user querystring.
 private String GetConnectionUser()
 {
 var name = Context.Headers[Constants.SignalR_HeaderID_Username];

 if (String.IsNullOrEmpty(name))
 {
 name = Context.QueryString[Constants.SignalR_HeaderID_Username];
 }
 if (String.IsNullOrEmpty(name))
 return null;

 // Notice that the partition keys are stored in azure storage as lower case
 return name.ToLowerInvariant();
 }

 public override Task OnConnected()
 {
 //var name = Context.User.Identity.Name;
 var name = this.GetConnectionUser();

 if(!String.IsNullOrEmpty(name))
 {
 var table = GetConnectionTable();
 var created = table.CreateIfNotExists();

 var entity = new ConnectionEntity(
 name.ToLower(),
 Context.ConnectionId);
 var insertOperation = TableOperation.InsertOrReplace(entity);
 table.Execute(insertOperation);
 }

 return base.OnConnected();
 }

 public override Task OnDisconnected(bool stopCalled)
 {
 //var name = Context.User.Identity.Name;
 var name = this.GetConnectionUser();

 if (!String.IsNullOrEmpty(name))
 {
 var table = GetConnectionTable();

 var deleteOperation = TableOperation.Delete(
 new ConnectionEntity(name, Context.ConnectionId) { ETag = "*" });
 table.Execute(deleteOperation);
 }

 return base.OnDisconnected(stopCalled);
 }

 private CloudTable GetConnectionTable()
 {

 var storageAccount =
 CloudStorageAccount.Parse(
 MyFitnessTrackerLibrary.Globals.MyFitAppSettings.AzureTableStorageConnectionString);
 var tableClient = storageAccount.CreateCloudTableClient();
 var table = tableClient.GetTableReference("connection");

 return table;
 }
 }
}

In my code example connections are stored and managed in Azure Table Storage(check the link below how to create one):
http://azure.microsoft.com/en-us/documentation/articles/storage-dotnet-how-to-use-tables/

When you connect to the hub you will enter the OnConnected() function. Here my implementation seeks for the username header value or QueryString username value(this is to just go around a problem in JavaScript client which I did not want to spend to much time on). After this we connect to the Azure Table Storage and add a new connection to the table.

When disconnecting the reverse is done to the Azure Storage Table.

The GetConnectionTable() function will open a connection to the storage table(check from azure management web console for your connection data).

The SignlarR has threee function which will send information to listening clients based on connection IDs:

  • Send
  • IsDataUpdateRequiredForWeb
  • IsDataUpdateRequiredForMobileClient

The SendMessageTo() function is used to get all of the connection for a user name which needs to be notified of updates.

The code is pretty simple an easy. Microsoft has done a great job documenting this: http://www.asp.net/signalr

Last thing which I recommend to do is to configure CORS support:

https://github.com/lionadi/MyFitnessTracker/blob/master/SignalRGateway/SignalRGateway/Startup.cs


// Branch the pipeline here for requests that start with "/signalr"
 app.Map("/signalr", map =>
 {
 // Setup the CORS middleware to run before SignalR.
 // By default this will allow all origins. You can
 // configure the set of origins and/or http verbs by
 // providing a cors options with a different policy.
 map.UseCors(CorsOptions.AllowAll);
 var hubConfiguration = new HubConfiguration
 {
 // You can enable JSONP by uncommenting line below.
 // JSONP requests are insecure but some older browsers (and some
 // versions of IE) require JSONP to work cross domain
 // EnableJSONP = true
 };
 // Run the SignalR pipeline. We're not using MapSignalR
 // since this branch already runs under the "/signalr"
 // path.
 map.RunSignalR(hubConfiguration);
 });

Simply copy&paste the code above to get it to work but notice that it will allow requests from all possible connections.
http://www.asp.net/signalr/overview/security/introduction-to-security#csrf

Connecting and listening to activities with the following clients:

You will need to import the following libraries in Visual Studio to get you clients to work on SignalR:

Microsoft ASP .NET SignalR .NET Client

Microsoft ASP .NET SignalR JavaScript Client

Connecting with the .NET Client

The .NET Client code is pretty easy to understand:


using System;
using System.Collections.Generic;
using System.Linq;
using System.Web;
using Microsoft.AspNet.SignalR.Client;
using System.Threading.Tasks;
using MyFitnessTrackerLibrary.ServiceBus;
using MyFitnessTrackerLibrary.Globals;

namespace MyFitnessTrackerLibrary.SignalRLogic
{
 public class HubGateway
 {

 private String hubLocation = MyFitAppSettings.SignalRHubHostLocation;
 private static HubGateway _hubGateway = null;
 private String hubProxyName = MyFitAppSettings.SignalRHubProxy;
 private IHubProxy hubProxy = null;
 private HubConnection hubConnection = null;
 private String sourceID = "NO ID";

 public IHubProxy HubProxyPoint
 {
 get { return this.hubProxy; }
 }

 public String SourceID
 {
 get
 {
 return this.sourceID;
 }

 set

 {
 this.sourceID = value;
 }
 }

 public HubGateway()
 {
 hubConnection = new HubConnection(this.hubLocation);
 hubProxy = hubConnection.CreateHubProxy(hubProxyName);
 }

 ~HubGateway()
 {
 this.Stop();
 }

 public async Task SendNormalMessage(String name, String message)
 {
 await this.Start(name);
 await this.HubProxyPoint.Invoke("Send", name, message + " #Source ID: " + this.sourceID);
 }

 public async Task IsDataUpdateRequiredForWeb(String name, bool isRequired, String message)
 {
 await this.Start(name);
 await this.HubProxyPoint.Invoke("IsDataUpdateRequiredForWeb", name, isRequired, message + " #Source ID: " + this.sourceID);
 await NotificationGateway.GetInstance().SendMessage("New data was added. Your UI is updated/updating.");
 }

 public async Task IsDataUpdateRequiredForMobileClient(String name, bool isRequired, String message)
 {
 await this.Start(name);
 await this.HubProxyPoint.Invoke("IsDataUpdateRequiredForMobileClient", name, isRequired, message + " #Source ID: " + this.sourceID);
 await NotificationGateway.GetInstance().SendMessage("New data was added. Your UI is updated/updating.");
 }

 public static HubGateway GetInstance()
 {
 if( _hubGateway == null)
 {
 _hubGateway = new HubGateway();
 }

 return _hubGateway;
 }

 public async Task Start(String userName)
 {
 if (hubConnection.State == ConnectionState.Disconnected)
 {
 if(!this.hubConnection.Headers.ContainsKey(Constants.SignalR_HeaderID_Username))
 this.hubConnection.Headers.Add(new KeyValuePair<string, string>(Constants.SignalR_HeaderID_Username, userName));

 await hubConnection.Start();
 }
 }

 public void Stop()
 {
 hubConnection.Stop();
 }
 }
}

The important part in this code in when you call the Invoke() function to invoke in the SignalR hub the needed function and notify registered clients of updates.

The second important part is the Start() function. We add the username data in the connection, this way the hub knows where to send the messages/updates requests. Yes I know there is a bug in the code above, only one user will receive messages from this piece of code. I haven’t got around to fix this in my original project but it’s a simple matter.

Connecting with the JavaScript client

</pre>
<pre>
var connection = $.hubConnection(Constants.SignalRGatewayLocation);
 connection.qs = { "username": CookieHelper.UserName };
 var contosoChatHubProxy = connection.createHubProxy(Constants.SignalRHubProxyName);

 contosoChatHubProxy.on(Constants.SignalRHubMethod_IsDataUpdateRequiredForWeb, function (name, isRequired, message) {
 // Html encode display name and message.
 var encodedName = $('<div />').text(name).html();
 //var encodedMsg = $('<div />').text("isDataUpdateRequiredForWeb is Update Required: " + isRequired + " Message: " + message).html();
 var encodedMsg = $('<div />').text("Updating UI. New data from the mobile app.").html();
 // Add the message to the page.
 $('#notifications').append('<ul><li><strong>' + encodedName
 + '</strong>:&nbsp;&nbsp;' + encodedMsg + '</li></ul>');
 highChartsController.LoadProperChartByUserSelection();
 });

 connection.start()
 .done(function () {
 console.log('Now connected, connection ID=' + connection.id
 );
 })
 .fail(function () {
 console.log('Could not connect');
 });

The code above is rather simple. You create a connection, define the hub name, register to a function on the SignalR hub and start the connection.
The only “weird” part is that the username is passed in the QueryString and not in the header. This was due to a problem which I could not fix and had to go around. There might be a better solution out there.

Connecting with the Android client

Now here comes the hard part. Getting SignalR to work on android was a pain in the but :). Lots of weird problems and lack or proper documentation.

To start with you will need this library downloaded and compiled in Android Studio(or some other Java development tool you are using).

https://github.com/SignalR/java-client

You also might need the following library:

http://java-websocket.org/

Then a good place to go next would be:

https://whathecode.wordpress.com/2014/03/20/getting-started-with-the-java-signalr-sdk/

BUT the example above did not work for me as it was stated in the example. Here what I had to do:

Add to your Android project these libraries:

signalr-client-sdk.jar

Java-WebSocket-1.3.0.jar

signalr-client-sdk-android-release.aar

The following libraries can be added from Android Studio UI: File -> Project Structure. Then add a new library from the plus icon and in the new popup select the “import .JAR or .AAR Package”.

Your gradle file should look something like this:

dependencies {
 compile fileTree(dir: 'libs', include: ['*.jar'])
 compile 'com.android.support:appcompat-v7:21.0.3'
 compile 'com.google.android.gms:play-services:6.5.87'
 compile 'com.google.code.gson:gson:2.3.1'
 compile files('libs/signalr-client-sdk.jar')
compile files('libs/notifications-1.0.1.jar')
compile files('libs/notification-hubs-0.4.jar')
 compile files('libs/Java-WebSocket-1.3.0.jar')
 compile project(':signalr-client-sdk-android-release')
}

The next step is to start to create a background service in Android that will be able to communicate with your desired activity.

We Start this by defining a interface which is implemented in the activity:

https://github.com/lionadi/MyFitnessTracker/blob/master/MyFitnessAndroidApp/app/src/main/java/com/example/adriansimionescu/myfitnessandroidapp/ServiceCallbacks.java


package com.example.adriansimionescu.myfitnessandroidapp;

public interface ServiceCallbacks {
    void updateUI();
}</pre>
<pre>// To implement it in your activity:</pre>
<pre>public class MainActivity extends ActionBarActivity implements ServiceCallbacks {
...
}

Next we create the background service:
https://github.com/lionadi/MyFitnessTracker/blob/master/MyFitnessAndroidApp/app/src/main/java/com/example/adriansimionescu/myfitnessandroidapp/SignalRService.java


package com.example.adriansimionescu.myfitnessandroidapp;

import android.app.IntentService;
import android.app.Service;
import android.content.Intent;
import android.os.Binder;
import android.os.IBinder;
import android.util.Log;
import android.widget.Toast;

import java.util.concurrent.ExecutionException;

import microsoft.aspnet.signalr.client.SignalRFuture;
import microsoft.aspnet.signalr.client.hubs.HubConnection;
import microsoft.aspnet.signalr.client.hubs.HubProxy;
import microsoft.aspnet.signalr.client.hubs.SubscriptionHandler1;
import microsoft.aspnet.signalr.client.hubs.SubscriptionHandler2;
import microsoft.aspnet.signalr.client.hubs.SubscriptionHandler3;
import microsoft.aspnet.signalr.client.transport.ClientTransport;
import microsoft.aspnet.signalr.client.transport.ServerSentEventsTransport;

public class SignalRService extends Service {

    // Binder given to clients
    private final IBinder binder = new LocalBinder();
    // Registered callbacks
    private ServiceCallbacks serviceCallbacks;

    // Class used for the client Binder.
    public class LocalBinder extends Binder {
        SignalRService getService() {
            // Return this instance of MyService so clients can call public methods
            return SignalRService.this;
        }
    }

    @Override
    public IBinder onBind(Intent intent) {
        return binder;
    }

    @Override
    public void onCreate() {
        super.onCreate();
    }

    public void setCallbacks(ServiceCallbacks callbacks) {
        this.serviceCallbacks = callbacks;
    }

    @SuppressWarnings("deprecation")
    @Override
    public void onStart(Intent intent, int startId) {
        super.onStart(intent, startId);
        Toast.makeText(this, "Service Start", Toast.LENGTH_LONG).show();

        String server = Constants.SignalRGateway;
        HubConnection connection = new HubConnection(server);
        connection.getHeaders().put("username", UserDataContainer.LoginData.userName);
        HubProxy proxy = connection.createHubProxy(Constants.SignalRHubName);

        //SignalRFuture<Void> awaitConnection = connection.start();

// This was added to get around a websocket problem with Android devices to the SignalR hub hosted in Azure
        ClientTransport transport = new ServerSentEventsTransport(connection.getLogger());

        SignalRFuture<Void> awaitConnection = connection.start(transport);
        try {
            awaitConnection.get();
            proxy.subscribe(this );
        } catch (InterruptedException e) {
            // TODO Auto-generated catch block
            e.printStackTrace();
        } catch (ExecutionException e) {
            // TODO Auto-generated catch block
            e.printStackTrace();
        }

        //--------------------------------------------------------------------------------
    }

    public void Send( String name, String message )
    {
        final String fmessage = message;
        final String fname = name;

    }

    public void IsDataUpdateRequiredForMobileClient( String name, boolean isRequired, String message ) {
        final String fmessage = message;
        final String fname = name;
        final boolean fisrequired = isRequired;
        if (serviceCallbacks != null) {
            serviceCallbacks.updateUI();
        }
    }

    @Override
    public void onDestroy() {
        super.onDestroy();
    }

}

There are a few important function.

onBind() => Use this to bind the actual instance of the service from your activity
setCallbacks() => use this to create a connection to the activity class interface so that we can call a desired method in the activity when a singalr message is received.

To bind to the SignalR function and message you need to define methods that use the same names as in the hub.
IsDataUpdateRequiredForMobileClient() and Send()

After the connection in made you need to call the subscribe method in the proxy class and pass the service class as a parameter. This will allow the binding between the defined methods above with the one in the SignalR hub.

The last part of the puzzle is that we call the interface function updateUI() which will trigger the same function in the activity to trigger and allow you to perform something in the activity.

Then all you have to do is to create the service instance in the activity, bind it and start it:
https://github.com/lionadi/MyFitnessTracker/blob/master/MyFitnessAndroidApp/app/src/main/java/com/example/adriansimionescu/myfitnessandroidapp/MainActivity.java

private SignalRService signalRService;</pre>
<pre>@Override
protected void onStart() {
    super.onStart();
    // Bind to LocalService

        Intent intent = new Intent(this, SignalRService.class);
        bindService(intent, serviceConnection, Context.BIND_AUTO_CREATE);
    // Do this to avoid starting multiple service, only one is needed
    if(this.signalRService == null) {
        this.startService(intent);
    }
}

@Override
protected void onStop() {
    super.onStop();

    // Unbind from service
    if (bound) {
        this.signalRService.setCallbacks(null); // unregister
        unbindService(serviceConnection);
        bound = false;
    }
}

/** Callbacks for service binding, passed to bindService() */
private ServiceConnection serviceConnection = new ServiceConnection() {

    @Override
    public void onServiceConnected(ComponentName className,
                                   IBinder service) {
        // cast the IBinder and get MyService instance
        SignalRService.LocalBinder binder = (SignalRService.LocalBinder) service;
        signalRService = binder.getService();
        bound = true;
        signalRService.setCallbacks(MainActivity.this); // register
    }

    @Override
    public void onServiceDisconnected(ComponentName arg0) {
        bound = false;
    }
};

/* Defined by ServiceCallbacks interface */
@Override
public void updateUI() {
// update your UI here
}

In the OnStart() and OnStop() functions we bind and start the service.

In the serviceConnection class instance we do the actual binding and notice how the code maps the activity to the service to be able to call the updateUI() function below.

 

The End

Huh, a long post but I hope you got the idea how to use notification hub, azure table storage and singnalr to communicate between different service and devices in different manner. SignalR is pretty cool what you can do with it. Especially with games and backend stuff. Cool 🙂

IIS Express and Android Studio connection issues

If you are getting a following like error when you are trying to connect to a Microsoft Web Service or whatever server point that is under IIS Express you might encounter the following error:

failed to connect to /127.0.0.1 (port “Your port number here”): connect failed: ECONNREFUSED (Connection refused)

The error above is due to a loopback problem. What you need to do is do some configurations.

  1. Find your applicationhost.config file under your documents(usually): C:\Users\”Your account name here”\Documents\IISExpress\config
  2. Then find your web service binding information see the example below(the sample in the green color, the red color binding is what is by default configured).
  3. Add the IP address and the port to which you want to connect at from android studio. The IP address has to be your development machines IP address.
  4. Also you need to add new rules to your windows firewall that allows your ports through which you wish the connection to be done.
  5. You may need to run Visual Studio in Administrator mode

<sites>

<site name=”MyFitnessTrackerWebAPI(1)” id=”17″>
<application path=”/” applicationPool=”Clr4IntegratedAppPool”>
<virtualDirectory path=”/” physicalPath=”C:\Users\Adrian Simionescu\Dropbox\MyFitnessTracker\MyFitnessTrackerWebAPI\MyFitnessTrackerWebAPI” />
</application>
<bindings>
<binding protocol=”http” bindingInformation=”*:52797:localhost” />
<binding protocol=”http” bindingInformation=”*:80:192.168.163.151″ />
<binding protocol=”http” bindingInformation=”*:52797:192.168.163.151″ />
</bindings>
</site>

</sites>

After this when you start up your webservice you should be able to access your service by address or host name. Also your IIS Express hosting information should look something in the this direction(based on the configuration above):

IIS express configurations