Hybrid Cloud: Type Safe Requests

The Hybrid Cloud

At VISUA, our workflow is very computational-intensive, so we approach the different cloud offerings available to us in a pragmatic way. We do this by leveraging the competition on the market to align with our client requirements in terms of speed, privacy, and cost. The result is a hybrid cloud approach that uses a combination of cloud providers with different APIs and specifications. This blog series will explore how we can unify the resources into one tool to manage the mental overhead of our engineers at VISUA (including myself!).

In this first post, we will explore the use of Haskell as a vessel for implementing this tool. Haskell is chosen as the language of implementation due to the type of safety and patterns this particular functional programming offers. It is also a language that I have become quite familiar with over the past few years.

Scaleway SDK

As a first step in unifying the diverse cloud providers I chose to write an SDK for one of them called Scaleway so that we can view this set of servers in our final implementation of the tool, and since it didn’t exist for Haskell, we had to roll our own. You can find the current implementation of the SDK here.

I’m going to focus on one particular aspect of the code instead of the whole codebase. Let’s look at how the requests can be made type-safe, preventing a user to provide the wrong kind of resource ID when retrieving a certain type of resource.

To start off, the request functions looked like the following snippets of code:

retrieveServer :: HeaderToken              -- | The X-Auth-Token 
               -> Region                   -- | The Region for the request
               -> ServerId                 -- | ID of the Server
               -> IO (Response ByteString) -- | Our raw result
retrieveServer' headerToken region (ServerId serverId) = do
  -- | Build the URL to point to the servers resource in the Scaleway API
  let url = unUrl (requestUrl region) <> "/" <> "servers" <> "/" <> (unpack serverId)
  -- | Create the params/headers for the GET request
      opts = defaults & (scalewayHeader headerToken)
  -- | Make the GET request
  getWith opts url

retrieveVolume :: HeaderToken
               -> Region
               -> VolumeId
               -> IO (Response ByteString)
retrieveVolume' headerToken region (VolumeId volumeId) = do
  let url = unUrl (requestUrl region) <> "/" <> "volumes" <> "/" <> (unpack volumeId)
      opts = defaults & (scalewayHeader headerToken)
  getWith opts url

There’s already too much duplication here, so let’s refactor this code a bit to make our code more concise.

type Resource = String  -- | Make Resource a type synonym for String

retrieveResource :: HeaderToken
                 -> Region
                 -> Resource    -- | Which resource we're asking for
                 -> resourceId  -- | Some type variable for the Resource ID
                 -> IO (Response ByteString)
retrieveResource headerToken region resource resourceId = do
  -- | Similar to before but we now have the resource passed in
  let url = unUrl (requestUrl region) <> "/" <> resource <> "/" <> undefined -- not sure how we can generalise on resourceId
      opts = defaults & (scalewayHeader headerToken)
  getWith opts url

Obviously we won’t get away with this because we have an undefined sitting there instead of our resourceId. The problem here is that we need to be able to get our ID from our Resource ID types.

To shed more light on the issue, our Resource ID types are defined as:

newtype ServerId = ServerId Text deriving (Show, Eq)
newtype VolumeId = VolumeId Text deriving (Show, Eq)

Typeclasses for Reusable Behavior

We need some way of unpacking the Text from our types here. To solve this, I came up with a typeclass that would give us the desired behaviour:

-- This will give us a way of extracting the Id out of our types
class HasResourceId f a where
  getResourceId :: f -> a

So let’s see what the instances for this typeclass will be:

instance HasResourceId ServerId Text where
  getResourceId (ServerId serverId) = serverId -- ServerId -> Text ~ f -> a

instance HasResourceId VolumeId Text where
  getResourceId (VolumeId volumeId) = volumeId -- VolumeId -> Text ~ f -> a

Now we can revisit our retrieveResource function:

-- so let's revisit our generalized retrieveResource
retrieveResource :: (HasResourceId resourceId Text) =>
                    HeaderToken
                 -> Region
                 -> Resource
                 -> resourceId
                 -> IO (Response ByteString)
retrieveResource headerToken region resource resourceId = do
  -- | `getResourceId resourceId` to get our ID out and `unpack` to get our Text type to String (getWith accepts a String for its URL parameter)
  let url = unUrl (requestUrl region) <> "/" <> resource <> "/" <> (unpack $ getResourceId resourceId)
      opts = defaults & (scalewayHeader headerToken)
  getWith opts url

Making our other retrieve functions even simpler!

-- now our retrieve definitions are simpler!
retrieveServer :: HeaderToken
               -> Region
               -> ServerId
               -> IO (Response ByteString)
retrieveServer headerToken region serverId = retrieveResource headerToken region "servers" serverId

retrieveVolume :: HeaderToken
               -> Region
               -> VolumeId
               -> IO (Response ByteString)
retrieveVolume headerToken region volumeId = retrieveResource headerToken region "volumes" volumeId

Type Safe Requests

The only thing is, Resource and resourceId have no way of relating to each other. For example, we could provide "volumes" as the Resource and give ServerId. In the same fell swoop we will get rid of the need to pass theResource value. To go about this we will introduce General Algebraic Data Types (GADTs) and the DataKinds extension.

{-# LANGUAGE DataKinds              #-}
{-# LANGUAGE GADTs                  #-}

-- | The enumeration of our resources
data ResourceType = ServerResource
                  | VolumeResource

-- ResourceType -> * means when declaring GET in a type signature we should
-- provide one of ServerResource or VolumeResource in the type constructor
data GET :: ResourceType -> * where
  ServerR :: ServerId -> GET ServerResource
  VolumeR :: VolumeId -> GET VolumeResource

We now have types that are explicitly related to the resources. So let us take a look at the types of our retrieve* functions once again:

retrieveServer :: HeaderToken
               -> Region
               -> GET ServerResource
               -> IO (Response ByteString)

retrieveVolume :: HeaderToken
              -> Region
              -> GET VolumeResource
              -> IO (Response ByteString)

If we recall from before retrieveResource had a type constraint HasResourceId resourceId Text so we need to make an instance for our new types:

instance HasResourceId (GET ServerResource) where
  getResourceId (ServerR serverId) = getResourceId serverId

instance HasResourceId (GET VolumeResource) where
  getResourceId (VolumeR volumeId) = getResourceId volumeId

We have gotten as far getting the ID for the resource, but we said we would also get rid of that pesky string value. Well here it is: typeclasses to the rescue again!

class HasResourceName f a where
  getResourceName   :: f -> a

instance HasResourceName (GET ServerResource) String where
  getResourceName _ = "servers"

instance HasResourceName (GET VolumeResource) String where
  getResourceName _ = "volumes"

Now we can write our final definition for retrieveResource and our other retrieve* functions:

retrieveResource :: (HasResourceId resource Text      -- | How we get the ID
                   , HasResourceName resource String) -- | How we get the resource path
                 => HeaderToken
                 -> Region
                 -> resource                          -- | Our polymorphic resource type
                 -> IO (Response ByteString)
retrieveResource headerToken region resource = do
  let url = unUrl (requestUrl region) <> "/" <> (getResourceName resource) <> "/" <> (unpack $ getResourceId resource)
      opts = defaults & (scalewayHeader headerToken)
  getWith opts url


retrieveServer :: HeaderToken
               -> Region
               -> GET ServerResource
               -> IO (Response ByteString)
retrieveServer headerToken region server = retrieveResource headerToken region server


retrieveVolume :: HeaderToken
               -> Region
               -> GET VolumeResource
               -> IO (Response ByteString)
retrieveVolume headerToken region volume = retrieveResource headerToken region volume

Our final touch is to add some smart constructors to construct our GET types:

mkGetServer :: Text -> GET ServerResource
mkGetServer = ServerR . ServerId

mkGetVolume :: Text -> GET VolumeResource
mkGetVolume = VolumeR . VolumeId

Conclusion

This solution gives us two benefits:

• We have completely abstracted our user-facing functions to only passing the necessary details i.e. the ID.
• We only allow the user to call retrieveVolume for a VolumeResource, meaning they cannot instantiate a ServerResource and call the wrong endpoint.

-- this compiles
main = do
  vol <- retrieveVolume "my-authentication" Paris (mkGetVolume "volume-id")
  print vol

-- this does not
main = do
  server <- retrieveServer "my-authentication" Paris (mkGetVolume "volume-id")
  print server

After that whirlwind of code, we have come to the end of this post. We touched lightly on the idea of a hybrid cloud and the pursuit of unifying this diverse land into one tool. We have also seen how Haskell can provide us with type safety by encoding our actions in the type system via GADTs and DataKinds.

Since I am no expert at Haskell, feel free to drop a line if you have any comments or suggestions (or even better: make a PR on the GitHub repo!)

In the next episode of this series, we will look at how we can dispel any fear of writing Haskell by exploring a tutorial on how to use Amazonka, the Haskell client library for Amazon Web Services. You can look forward to exploring documentation and becoming more comfortable with the GHCI REPL. The tutorial will show how we can make requests using Amazonka and convert the results into our own domain type, in this case, a VISUA Instance.

For more information about Haskell you can check out https://www.haskell.org/, https://www.fpcomplete.com/haskell, http://www.haskellbook.com, or swing by http://www.fpchat.com/.