Everything Else

Hybrid Cloud: Type Safe Requests
BLOG

Hybrid Cloud: Type Safe Requests

Reading Time: 6 minutes

The Hybrid Cloud

At VISUA, our workflow is very computational-intensive, so we approach the different cloud offerings available to us in a pragmatic way. We do this by leveraging the competition on the market to align with our client requirements in terms of speed, privacy, and cost. The result is a hybrid cloud approach that uses a combination of cloud providers with different APIs and specifications. This blog series will explore how we can unify the resources into one tool to manage the mental overhead of our engineers at VISUA (including myself!).

In this first post, we will explore the use of Haskell as a vessel for implementing this tool. Haskell is chosen as the language of implementation due to the type of safety and patterns this particular functional programming offers. It is also a language that I have become quite familiar with over the past few years.

Scaleway SDK

As a first step in unifying the diverse cloud providers I chose to write an SDK for one of them called Scaleway so that we can view this set of servers in our final implementation of the tool, and since it didn’t exist for Haskell, we had to roll our own. You can find the current implementation of the SDK here.

I’m going to focus on one particular aspect of the code instead of the whole codebase. Let’s look at how the requests can be made type-safe, preventing a user to provide the wrong kind of resource ID when retrieving a certain type of resource.

To start off, the request functions looked like the following snippets of code:

retrieveServer :: HeaderToken              -- | The X-Auth-Token 
               -> Region                   -- | The Region for the request
               -> ServerId                 -- | ID of the Server
               -> IO (Response ByteString) -- | Our raw result
retrieveServer' headerToken region (ServerId serverId) = do
  -- | Build the URL to point to the servers resource in the Scaleway API
  let url = unUrl (requestUrl region) <> "/" <> "servers" <> "/" <> (unpack serverId)
  -- | Create the params/headers for the GET request
      opts = defaults & (scalewayHeader headerToken)
  -- | Make the GET request
  getWith opts url

retrieveVolume :: HeaderToken
               -> Region
               -> VolumeId
               -> IO (Response ByteString)
retrieveVolume' headerToken region (VolumeId volumeId) = do
  let url = unUrl (requestUrl region) <> "/" <> "volumes" <> "/" <> (unpack volumeId)
      opts = defaults & (scalewayHeader headerToken)
  getWith opts url
Language

There’s already too much duplication here, so let’s refactor this code a bit to make our code more concise.

type Resource = String  -- | Make Resource a type synonym for String

retrieveResource :: HeaderToken
                 -> Region
                 -> Resource    -- | Which resource we're asking for
                 -> resourceId  -- | Some type variable for the Resource ID
                 -> IO (Response ByteString)
retrieveResource headerToken region resource resourceId = do
  -- | Similar to before but we now have the resource passed in
  let url = unUrl (requestUrl region) <> "/" <> resource <> "/" <> undefined -- not sure how we can generalise on resourceId
      opts = defaults & (scalewayHeader headerToken)
  getWith opts url
Language

Obviously we won’t get away with this because we have an undefined sitting there instead of our resourceId. The problem here is that we need to be able to get our ID from our Resource ID types.

To shed more light on the issue, our Resource ID types are defined as:

newtype ServerId = ServerId Text deriving (Show, Eq)
newtype VolumeId = VolumeId Text deriving (Show, Eq)
Language

Typeclasses for Reusable Behavior

We need some way of unpacking the Text from our types here. To solve this, I came up with a typeclass that would give us the desired behaviour:

-- This will give us a way of extracting the Id out of our types
class HasResourceId f a where
  getResourceId :: f -> a
Language

So let’s see what the instances for this typeclass will be:

instance HasResourceId ServerId Text where
  getResourceId (ServerId serverId) = serverId -- ServerId -> Text ~ f -> a

instance HasResourceId VolumeId Text where
  getResourceId (VolumeId volumeId) = volumeId -- VolumeId -> Text ~ f -> a
Language

Now we can revisit our retrieveResource function:

-- so let's revisit our generalized retrieveResource
retrieveResource :: (HasResourceId resourceId Text) =>
                    HeaderToken
                 -> Region
                 -> Resource
                 -> resourceId
                 -> IO (Response ByteString)
retrieveResource headerToken region resource resourceId = do
  -- | `getResourceId resourceId` to get our ID out and `unpack` to get our Text type to String (getWith accepts a String for its URL parameter)
  let url = unUrl (requestUrl region) <> "/" <> resource <> "/" <> (unpack $ getResourceId resourceId)
      opts = defaults & (scalewayHeader headerToken)
  getWith opts url
Language

Making our other retrieve functions even simpler!

-- now our retrieve definitions are simpler!
retrieveServer :: HeaderToken
               -> Region
               -> ServerId
               -> IO (Response ByteString)
retrieveServer headerToken region serverId = retrieveResource headerToken region "servers" serverId

retrieveVolume :: HeaderToken
               -> Region
               -> VolumeId
               -> IO (Response ByteString)
retrieveVolume headerToken region volumeId = retrieveResource headerToken region "volumes" volumeId
Language

Type Safe Requests

The only thing is, Resource and resourceId have no way of relating to each other. For example, we could provide "volumes" as the Resource and give ServerId. In the same fell swoop we will get rid of the need to pass theResource value. To go about this we will introduce General Algebraic Data Types (GADTs) and the DataKinds extension.

{-# LANGUAGE DataKinds              #-}
{-# LANGUAGE GADTs                  #-}

-- | The enumeration of our resources
data ResourceType = ServerResource
                  | VolumeResource

-- ResourceType -> * means when declaring GET in a type signature we should
-- provide one of ServerResource or VolumeResource in the type constructor
data GET :: ResourceType -> * where
  ServerR :: ServerId -> GET ServerResource
  VolumeR :: VolumeId -> GET VolumeResource
Language

We now have types that are explicitly related to the resources. So let us take a look at the types of our retrieve* functions once again:

retrieveServer :: HeaderToken
               -> Region
               -> GET ServerResource
               -> IO (Response ByteString)

retrieveVolume :: HeaderToken
              -> Region
              -> GET VolumeResource
              -> IO (Response ByteString)
Language

If we recall from before retrieveResource had a type constraint HasResourceId resourceId Text so we need to make an instance for our new types:

instance HasResourceId (GET ServerResource) where
  getResourceId (ServerR serverId) = getResourceId serverId

instance HasResourceId (GET VolumeResource) where
  getResourceId (VolumeR volumeId) = getResourceId volumeId
Language

We have gotten as far getting the ID for the resource, but we said we would also get rid of that pesky string value. Well here it is: typeclasses to the rescue again!

class HasResourceName f a where
  getResourceName   :: f -> a

instance HasResourceName (GET ServerResource) String where
  getResourceName _ = "servers"

instance HasResourceName (GET VolumeResource) String where
  getResourceName _ = "volumes"
Language

Now we can write our final definition for retrieveResource and our other retrieve* functions:


retrieveResource :: (HasResourceId resource Text      -- | How we get the ID
                   , HasResourceName resource String) -- | How we get the resource path
                 => HeaderToken
                 -> Region
                 -> resource                          -- | Our polymorphic resource type
                 -> IO (Response ByteString)
retrieveResource headerToken region resource = do
  let url = unUrl (requestUrl region) <> "/" <> (getResourceName resource) <> "/" <> (unpack $ getResourceId resource)
      opts = defaults & (scalewayHeader headerToken)
  getWith opts url


retrieveServer :: HeaderToken
               -> Region
               -> GET ServerResource
               -> IO (Response ByteString)
retrieveServer headerToken region server = retrieveResource headerToken region server


retrieveVolume :: HeaderToken
               -> Region
               -> GET VolumeResource
               -> IO (Response ByteString)
retrieveVolume headerToken region volume = retrieveResource headerToken region volume
Language

Our final touch is to add some smart constructors to construct our GET types:

mkGetServer :: Text -> GET ServerResource
mkGetServer = ServerR . ServerId

mkGetVolume :: Text -> GET VolumeResource
mkGetVolume = VolumeR . VolumeId
Language

Conclusion

This solution gives us two benefits:

  • We have completely abstracted our user-facing functions to only passing the necessary details i.e. the ID.
  • We only allow the user to call retrieveVolume for a VolumeResource, meaning they cannot instantiate a ServerResource and call the wrong endpoint.
-- this compiles
main = do
  vol <- retrieveVolume "my-authentication" Paris (mkGetVolume "volume-id")
  print vol

-- this does not
main = do
  server <- retrieveServer "my-authentication" Paris (mkGetVolume "volume-id")
  print server
Language

After that whirlwind of code, we have come to the end of this post. We touched lightly on the idea of a hybrid cloud and the pursuit of unifying this diverse land into one tool. We have also seen how Haskell can provide us with type safety by encoding our actions in the type system via GADTs and DataKinds.

Since I am no expert at Haskell, feel free to drop a line if you have any comments or suggestions (or even better: make a PR on the GitHub repo!)

In the next episode of this series, we will look at how we can dispel any fear of writing Haskell by exploring a tutorial on how to use Amazonka, the Haskell client library for Amazon Web Services. You can look forward to exploring documentation and becoming more comfortable with the GHCI REPL. The tutorial will show how we can make requests using Amazonka and convert the results into our own domain type, in this case, a VISUA Instance.

For more information about Haskell you can check out https://www.haskell.org/https://www.fpcomplete.com/haskellhttp://www.haskellbook.com, or swing by http://www.fpchat.com/.

Book A Demo

RELATED

  • The visual AI People Video: We Are VISUA - Discover why we’re the Visual-AI people - delivering cutting-edge computer vision solutions ranging from logo/mark detection and counterfeit product detection to holographic authentication and phishing detection... Posted in: Everything Else
  • shopping on phone 4 Ecommerce Trends – Updated for 2022 - Reading Time: 3 minutes As 2022 is well underway, new trends are beginning to emerge within the world of ecommerce. The online retail industry saw an […] Posted in: Everything Else
  • Visual-AI Logo detection pepsi-cola logo 8 Visual-AI FAQs - Reading Time: 5 minutes 1. What is Visual-AI? Visual-AI is the identification of objects and logos within an image. When using Visual-AI technology, images or videos […] Posted in: Everything Else
BLOG BLOG
Could Computer Vision be the Solution to the International Luggage Issue?

Reading Time: 4 minutes The explosion in world travel after the easing of lockdowns and travel restrictions has seen airports struggle to cope. One of the […]

Everything Else Technology
BLOG BLOG
Franco De Bonis Interview With Private Internet Access

Reading Time: < 1 minute VISUA Marketing Director Franco De Bonis sat down to chat with Private Internet Access recently to discuss our offering. Franco details the […]

Everything Else
BLOG BLOG
The History of the Logo

Reading Time: 4 minutes The beginning of logo design We all know that a logo has the potential to be a very powerful asset for a […]

Everything Else

Trusted by the world's leading platforms, marketplaces and agencies

Integrate Visual-AI Into Your Platform

Seamlessly integrating our API is quick and easy, and if you have questions, there are real people here to help. So start today; complete the contact form and our team will get straight back to you.

  • This field is for validation purposes and should be left unchanged.